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EXECUTIVE  SUMMARY 


Instrument  approach  procedure  (lAP)  charts  can  be  densely  packed  with  information.  This 
high  information  density  can  make  information  difficult  to  find,  particularly  in  a  poorly  lit 
cockpit  during  turbulence.  The  workload  imposed  by  operations  in  the  terminal  area 
compounds  the  legibility  problem  by  reducing  the  amount  of  time  pilots  have  to  search  for 
information,  and  by  reducing  the  amount  of  attention  resources  pilots  can  focus  on  chart 
interpretation. 

The  Volpe  Center's  Cockpit  Human  Factors  Program  conducted  a  series  of  evaluations  to 
format  lAP  chart  information  to  more  closely  conform  to  the  way  pilots  actually  use  the 
information.  All  of  this  work  has  contributed  to  the  evolution  of  the  Volpe  prototype  lAP 
chart  format.  The  prototype's  major  design  features  are  the  briefing  strip  and  iconic  missed 
^proach  procedure  instructions.  The  briefing  strip  consists  of  three  rows  of  tabularized 
information  at  the  top  of  the  chart.  This  feature  is  designed  primarily  for  use  in  preparing  for 
the  approach.  Each  information  element  is  given  in  the  order  in  which  it  will  be  used.  The 
pilot  no  longer  has  to  search  through  the  entire  chart  to  assemble  this  data.  In  the  profile 
view,  the  "up  and  out"  portion  of  the  missed  approach  instructions  is  depicted  in  icons  rather 
than  text.  These  icons  tell  the  pilot  all  that  is  required  to  get  the  plane  up  and  out  -  and  this 
critical  information  is  more  easily  located  than  if  it  were  embedded  in  text. 

The  purpose  of  the  prototype  is  to  improve  the  speed  and  accuracy  with  which  pilots  find 
information  on  lAP  charts.  The  objective  of  this  study  was  to  determine  if  the  prototype  lAP 
chart  format  would  allow  pilots  to  find  information  faster  and  more  accurately  during  actual 
flight. 

Ten  licensed  pilots  rated  for  instrument  flight  participated  as  subjects  in  this  experiment.  Each 
of  the  approaches  were  depicted  in  two  chart  formats:  National  Ocean  Service  (NOS)  and  the 
Volpe  prototype.  Each  approach  was  based  on  an  actual  approach  taken  from  the  U.S. 
Terminal  Procedures  charts  published  by  the  U.S.  National  Oceanic  and  Atmospheric 
Administration.  Non-precision  approaches  from  around  the  United  States  were  selected  for  the 
study.  Global  Positioning  System  (GPS)  overlay  approaches  were  constructed  for  the  airport 
where  the  study  was  conducted  because  it  was  not  possible  to  use  actual  VOR  or  NDB 
approaches  for  this  experiment. 

Pilots  participated  for  two  days.  The  first  day  consisted  of  training  in  a  simulator,  during 
which  the  pilots  gained  experience  in  using  the  prototype  chart  format,  flying  with  guidance 
from  a  GPS  receiver,  and  having  someone  ask  them  a  series  of  questions  while  flying  an 
approach.  Data  collection  during  actual  flight  occurred  on  the  second  day.  Both  days  were 
composed  of  a  morning  session  and  an  afternoon  session.  In  each  session,  pilots  were  exposed 
to  only  one  chart  format  (NOS  or  prototype).  All  pilots  received  training  for  and  flew  with 
both  chart  formats. 

Pilots  took  advantage  of  the  prototype's  briefing  strip  to  search  for  information  to  answer 
questions.  They  found  information  faster  on  the  prototype  chart  than  on  the  NOS  chart.  No 
difference  was  found  between  the  accuracy  of  the  answers  given  when  pilots  used  the 


XI 


prototype  and  when  they  used  the  NOS  format.  Pilots  found  information  regarding  the  "up 
and  out"  portion  of  the  missed  approach  instructions  faster  when  they  used  the  iconic  missed 
approach  instructions  on  the  prototype  chart  than  when  they  used  text  instructions  on  the  NOS 
chart. 

Most  pilots  preferred  the  prototype  format  for  studying  for  the  approach.  All  pilots  preferred 
the  prototype  format  for  executing  the  missed  approach  because  of  the  iconic  instructions. 
None  of  the  pilots  preferred  the  NOS  format  for  executing  any  approach  phase. 

These  findings  are  consistent  with  the  outcome  of  past  laboratory  experimentation,  subject 
matter  expert  interviews,  a  review  of  the  relevant  literature,  and  a  field  evaluation  using 
simulators.  The  National  Ocean  Service  is  strongly  urged  to  adopt  the  prototype  format  for  its 
lAP  charts. 
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1.  INTRODUCTION 


1.1  THE  PROBLEM 

Instrument  approach  procedure  (lAP)  charts  provide  pilots  with  the  information  required  to  fly 
an  instrument  approach  procedure  to  a  runway  of  intended  landing,  and  if  necessary,  to  fly  the 
missed  approach  procedure  (International  Civil  Aviation  Organization,  1985).  The  U.S. 
Department  of  Commerce  National  Ocean  Service  (NOS)  is  a  major  manufacturer  of  lAP 
charts.  An  example  of  a  NOS  lAP  chart,  the  instrument  landing  system  approach  to  runway 
25  at  Los  Angeles  International  Airport,  is  given  in  Figure  1. 

As  this  figure  illustrates,  lAP  charts  can  be  densely  packed  with  information.  This  high 
information  density  can  make  information  difficult  to  find,  particularly  in  a  poorly  lit  cockpit 
during  turbulence.  The  workload  imposed  by  operations  in  the  terminal  area  compounds  the 
legibility  problem  by  reducing  the  amount  of  time  pilots  have  to  search  for  information,  and 
by  reducing  the  amount  of  attention  resources  pilots  can  focus  on  chart  interpretation.  Cox 
and  Connor  (1987)  surveyed  pilots  regarding  human  factors  problems  they  have  encountered 
with  lAP  charts.  The  authors  stated  that  "Inefficient  information  transfer  techniques  used  in 
charting  approach,  missed  approach,  and  instrument  departure  procedures  and  associated 
information  are  causing  ...  excessive  head-in-the-cockpit  time  ...  that  could  have  ...  adverse 
impact  on  air  safety  ..."  (p.  11). 


1.2  HOW  THE  VOLPE  PROTOTYPE  lAP  CHART  FORMAT  WAS  DEVELOPED 

The  Volpe  Center's  Cockpit  Human  Factors  Program  conducted  a  series  of  evaluations  of  both 
individual  chart  components  and  the  Volpe  prototype  as  a  whole.  The  goal  was  to  format  the 
information  to  more  closely  conform  to  the  way  pilots  actually  use  the  information.  This  work 
has  been  accomplished  by  taking  four  approaches:  laboratory  experimentation,  subject  matter 
expert  evaluations,  a  review  of  the  relevant  literature,  and  a  field  evaluation  conducted  at  the 
training  centers  of  four  airlines. 

Multer,  Warner,  DiSario,  and  Himtley  (1991)  conducted  two  studies  in  lAP  chart  design.  The 
first  study  examined  three  different  methods  of  presenting  the  final  approach  course:  bolding 
(increasing  the  line  weight  of  the  text),  boxing,  and  reverse  contrast  (white  text  in  a  black 
box).  Both  boxing  and  reverse  contrast  improved  the  speed  with  which  pilots  could  find  items 
on  lAP  charts.  Reverse  contrast  was  so  compelling  that  the  authors  stated  that  it  might  detract 
from  a  pilot's  search  performance  if  the  item  printed  in  reverse  contrast  was  not  the  item 
being  searched  for.  Therefore,  the  prototype  highlights  the  final  approach  course  by  boxing  it. 

The  second  study  conducted  by  Multer,  et.  al.  addressed  the  effects  of  four  different  layouts 
of  radio  frequencies.  Text  size  and  type  of  font  were  held  constant.  The  four  layouts  were: 
frequency  under  the  name  (similar  to  current  NOS  format);  frequency  to  the  right  of  the  name 
(similar  to  current  Jeppesen  format);  frequency  to  the  right  of  the  name,  with  the  name  and 
frequencies  in  their  own  left-justified  columns;  and  frequency  under  the  name  with  both 
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boxed  and  centered  within  the  box  (similar  to  Canadian  Department  of  Energy,  Mines  and 
Resources  lAP  charts).  Pilots  found  the  desired  frequencies  faster  with  the  two  column  and 
boxed  layouts  than  with  the  other  two  layouts.  The  prototype  format  uses  the  boxed  and 
centered  method  to  format  the  radio  frequencies. 

Osborne  and  Huntley  (1992)  conducted  an  experiment  to  determine  if  encoding  the  missed 
approach  instructions  in  icons  would  allow  pilots  to  comprehend  the  instructions  faster  and 
more  accurately  than  text  instructions.  They  also  examined  the  effect  of  varying  the  amount 
of  information  (low,  medium,  or  high)  in  the  instructions  on  comprehension  speed  and 
accuracy.  Across  the  range  of  information  content  levels,  iconic  missed  approach  instructions 
were  comprehended  more  quickly  and  as  accurately  as  instructions  coded  in  text  of  the  font 
style  and  size  used  by  NOS.  Regardless  of  encoding  method,  report  accuracy  was  worse  for 
instructions  with  a  high  information  content  level.  These  results  were  implemented  in  the 
prototype  format  by  using  icons  to  code  the  initial  ("up  and  out")  portion  of  the  missed 
approach  instructions. 

Throughout  its  development,  the  prototype  lAP  chart  has  been  reviewed  by  subject  matter 
experts.  These  reviews  have  been  conducted  both  by  individuals,  and  aeronautical  charting 
interest  groups  such  as  the  FAA  Govemment/Industry  Aeronautical  Charting  Forum,  Air 
Transport  Association  Charting  and  Data  Display  Working  Group  (ATA  CDDWG),  and 
Society  of  Automotive  Engineers  G-10  Aeronautical  Charting  Subcommittee.  The  membership 
of  these  professional  organizations  include  terminal  instrument  procedures  experts,  commercial 
air  carrier  pilots  and  management,  commercial  and  U.S.  Government  cartographers  and  chart 
manufacturers,  and  military  and  general  aviation  representatives.  Several  of  the  critiques 
offered  by  these  experts  have  been  integrated  into  the  prototype  format.  In  addition  to  these 
committee  reviews,  several  in-depth  reviews  of  the  charts  have  been  conducted  with 
individual  pilots,  cartographers,  and  other  subject  matter  experts  regarding  human  factors 
problems  with  lAP  chart  design  (Osborne  and  Huntley,  1993).  This  continuous  process  of 
expert  review  and  refinement  has  evolved  the  prototype  format  to  its  current  state. 

Mangold,  Eldredge,  and  Lauber  (1992)  conducted  an  extensive  literature  review  in  the  areas 
of  human  factors,  cartography,  visual  psychophysics,  reading,  and  information  formatting. 
General  guidance  information  was  then  extracted  from  these  references  and  organized  into  a 
set  of  design  tools  a  cartographer  could  use  to  help  make  formatting  decisions.  As  changes 
were  recommended  during  the  evolution  of  the  prototype's  design,  this  handbook  was  used  as 
a  reference  source  to  verify  that  the  suggested  change  did  not  conflict  with  the  available 
guidance. 

Blomberg,  Bishop,  and  Hamilton  (in  press)  conducted  a  field  study  for  the  Volpe  Center 
Cockpit  Human  Factors  Program  to  determine  the  preferences  of  Part  121  air  carrier  pilots  for 
alternative  lAP  chart  designs.  The  prototype  format  which  was  originally  developed  for 
implementation  with  NOS  lAP  charts  was  adapted  for  Jeppesen  style  lAP  charts.  This 
adaptation  was  accomplished  by  Volpe  working  in  cooperation  with  ATA  CDDWG  to 
produce  a  prototype  format  tailored  for  Part  121  carrier  use.  Jeppesen  also  produced  a 
prototype  version  of  their  chart.  These  two  prototypes  were  tested  against  the  current  Jeppesen 
format  in  an  operationally  realistic  environment. 
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Pilots  were  asked  to  give  their  opinions  on  the  two  prototypes  and  standard  Jeppesen  lAP 
formats  after  flying  one  of  the  prototypes  and  the  current  format  in  a  full  motion  conventional 
or  glass  cockpit  aircraft  simulator.  Thirty  seven  pilots  flew  the  Volpe/ATA  prototype  and  44 
other  pilots  flew  the  Jeppesen  prototype.  All  81  pilots  flew  the  current  format  as  well.  Ten 
other  pilots  did  not  fly  any  charts  but  were  briefed  on  the  formats  and  asked  for  their 
opinions.  A  total  of  over  400  approaches  were  flown  at  the  training  centers  of  four 
commercial  air  carriers. 

Pilots  preferred  selected  features  from  both  prototypes  over  the  current  Jeppesen  format. 

These  features  were  combined  to  produce  the  composite  prototype  shown  in  Figure  2.  As 
evidenced  by  this  figure,  the  major  features  of  the  original  NOS  prototype  remained  intact. 
When  pilots  were  asked  to  choose  which  chart  they  would  buy  for  their  company  to  use,  90 
out  of  the  91  pilots  chose  one  of  two  prototypes.  Fifty  nine  per  cent  of  the  pilots  chose  the 
Volpe/ATA  prototype.  This  number  was  less  than  expected  because  pilots  exhibited  a  strong 
bias  for  preferring  the  prototype  they  flew  with.  Even  so,  while  41%  of  the  pilots  who  flew 
with  the  Jeppesen  prototype  picked  the  Volpe/ATA  prototype,  only  16%  of  the  pilots  who 
flew  with  the  Volpe/ATA  prototype  crossed  over  and  picked  the  Jeppesen  prototype.  The 
comments  indicated  that  having  a  chance  to  use  the  briefing  strip  and  missed  approach  icons 
during  simulated  flight  convinced  pilots  that  these  features  allowed  them  to  find  the 
information  faster.  Pilots  did  not  expect  any  problems  in  implementing  a  prototype  chart  on 
the  flight  line. 


1.3  PROTOTYPE  lAP  CHART  FORMAT 

All  of  the  work  described  above  has  contributed  to  the  evolution  of  a  prototype  NOS  lAP 
chart  format.  This  design  that  was  originally  developed  in  the  laboratory  for  NOS  lAP  charts 
and  refined  during  field  testing  with  Jeppesen  lAP  charts  and  subject  matter  expert  reviews  is 
shown  in  Figure  3.  The  purpose  of  the  prototype  is  to  improve  the  speed  and  accuracy  with 
which  pilots  locate  and  comprehend  information  on  lAP  charts. 

1.3.1  Briefing  Strip 

These  three  rows  of  information  are  used  for  preparing  for  the  approach.  Each  information 
element  is  given  in  the  order  in  which  it  will  be  used.  In  the  first  and  third  lines  of  the  strip, 
the  information  to  be  briefed  is  given  in  bold  type,  while  the  name  of  the  information  is 
shown  in  regular  type.  The  pilot  no  longer  has  to  search  through  the  entire  chart  to  assemble 
this  data. 

1.3. 1.1  -  The  information  required  for  quick  reference  is  in  the  top  row.  The  concept  of 
formatting  this  safety  critical  information  (particularly  the  first  three  items)  grew  out  of 
subject  matter  expert  interviews  (Osborne  and  Huntley,  1993).  The  first  box  contains  the  type, 
identifier,  and  frequency  of  the  navaid  that  serves  this  approach.  The  final  approach  course  is 
given  in  the  second  box.  The  third  box  contains  the  final  approach  fix  (FAF)  name  and  the 
altitude  at  the  FAF.  The  fourth  box  shows  the  touchdown  zone  elevation  (TDZE).  If  the 
charted  procedure  were  a  non-precision  approach  to  either  of  two  runways,  both  TDZEs 
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would  be  shown,  and  both  would  be  bolded.  The  airport  elevation  is  given  in  the  fifth  box. 

To  the  right  of  this  row,  the  four-letter  ICAO  airport  identifier  is  shown  in  italics. 

1.3. 1.2  -  The  second  row  contains  all  equipment  and  procedural  notes  (navigation  notes 
remain  in  the  profile  section),  an  approach  lighting  system  sketch,  and  missed  approach 
instructions.  If  any  visual  glideslope  indicators  are  present,  they  are  also  depicted  in  the 
approach  lighting  system  sketch.  If  a  parallel  nmway  has  the  same  lighting  system,  an 
italicized  note  is  given  in  the  notes  box.  Locating  all  equipment  and  procedural  notes  in  one 
location  helps  pilots  to  find  the  notes  they  are  looking  for,  as  well  as  notice  the  notes  that 
might  previously  have  been  unnoticed  in  a  densely  packed  plan  view.  These  notes  have  been 
moved  up  to  the  top  of  the  chart  to  make  them  easier  to  read  in  poor  lighting.  The  approach 
lighting  sketch  was  initially  recommended  by  a  Part  121  flight  crew  during  a  jumpseat 
interview  (Osborne  and  Huntley,  1993).  This  sketch  received  strong  support  during  field 
testing  (Blomberg,  Bishop,  and  Hamilton,  in  press). 

1.3. 1.3  -  The  results  of  research  conducted  by  Multer,  Warner,  DiSario,  and  Huntley  (1991) 
on  communication  frequency  layout  are  implemented  in  the  third  row  of  the  briefing  strip. 
The  ATIS  and  communication  frequencies  are  listed  in  order  of  their  use  and  the  numbers  are 
always  shown  under  the  labels.  In  order  to  save  space,  they  are  not  as  large  as  the  numbers 
listed  above.  They  are  made  available  here  for  verification,  early  radio  set-up,  or  in  case  of 
blockage  of  an  active  communication  channel.  Both  general  aviation  and  air  carrier  pilots 
recommended  that  since  the  plan  view  is  oriented  North-up,  any  West  and  East  frequencies 
should  be  placed  on  the  left  and  right,  respectively  (Osborne  and  Huntley,  1993). 

1.3.2  MSA  Circle 

The  MSA  circle  will  float  within  the  plan  view  to  occupy  unused  space. 

1.3.2. 1  -  The  text  identification  for  the  reference  navigational  aid  (navaid)  is  given  in  the 
center  of  the  circle  (Osborne  and  Huntley,  1993). 

1.3.2.2  -  The  sectors  of  the  circle  are  defined  by  radials  rather  than  bearings,  since  pilots 
usually  think  in  terms  of  radials  rather  than  bearings  to  the  station  (Osborne  and  Huntley, 
1993). 

1.3.3  Plan  View 

1.3.3. 1  -  The  approach  course  number  has  been  boxed  to  enhance  detectability  and  readability 
(Multer,  Warner,  DiSario,  and  Huntley,  1991). 

1.3. 3. 2  -  The  name  of  the  missed  approach  holding  fix  has  been  bolded  (Mangold,  Eldredge, 
and  Lauber,  1992). 

1.3. 3.3  -  The  enroute  facilities  and  feeder  facilities  rings  have  been  removed  (Osborne  and 
Huntley,  1993). 
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1.3.4  Profile  View 


1.3.4. 1  -  In  the  profile  view,  the  "up  and  out"  portion  of  the  missed  approach  instructions  is 
depicted  in  icons  rather  than  text.  These  icons  tell  the  pilot  all  that  is  required  to  get  the  plane 
up  and  out  -  and  this  critical  information  is  more  easily  located  than  if  it  were  embedded  in 
text.  The  first  altitude  and  navaid  frequency  are  emphasized  by  bolding.  If  there  was  a 
heading  in  the  first  box,  it  too  would  be  bolded.  Osborne  and  Huntley  (1992)  demonstrated 
that  encoding  the  missed  approach  instructions  in  icons  allows  pilots  to  comprehend  the 
instructions  faster  than  text  instructions. 

1.3. 4. 2  -  The  approach  course  number  has  been  boxed  here  as  well  (Multer,  Warner,  DiSario, 
and  Huntley,  1991). 

1.3.5  Minimums 

1.3.5. 1  -  Bold  type  has  been  used  to  make  the  MDAs  easier  to  find  and  read  (Mangold, 
Eldredge,  and  Lauber,  1992). 


1.4  PRESENT  RESEARCH  ISSUES 

Although  individual  components  of  the  prototype  had  been  tested  in  the  laboratory,  and  the 
format  as  a  whole  had  been  applied  to  a  Jeppesen  style  lAP  chart  and  field  tested,  another 
evaluation  was  needed  to  determine  the  effects  of  applying  the  format  to  an  NOS  chart. 
Measurements  of  the  search  time  required  by  pilots  to  find  specific  information  items,  as  well 
as  the  accuracy  of  their  search  during  actual  flight  were  needed  to  determine  if  the  prototype 
format  would  allow  faster  and  more  accurate  information  retrieval  than  the  current  NOS 
format. 


1.5  OBJECTIVE  AND  HYPOTHESES 

The  objective  of  this  study  was  to  determine  if  the  prototype  lAP  chart  format  would  allow 
pilots  to  find  information  faster  and  more  accurately  during  actual  flight.  The  specific 
hypotheses  were: 

•  The  formatting  of  the  briefing  strip  would  result  in  faster  and  more  accurate 
retrieval  of  the  information  it  contained  than  the  current  NOS  formatting  of  that 
information. 

•  Iconic  missed  approach  instructions  would  produce  faster  and  more  accurate 
information  retrieval  than  text  instructions. 

•  Overall,  the  prototype  chart  would  produce  faster  and  more  accurate 
information  retrieval  than  the  NOS  chart. 

•  Pilots  would  prefer  the  prototype  format  to  the  NOS  format. 
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2.  METHOD 


2.1  SUBJECTS 

Ten  licensed  pilots  rated  for  flight  during  instrument  flight  rules  (IFR)  conditions  participated 
as  subjects  in  this  experiment.  Pilots  were  recruited  from  a  subject  pilot  database  maintained 
at  the  Center  for  Human  Factors  Research  in  Transportation  at  the  Volpe  Center.  Pilots  were 
paid  ten  dollars  per  hour  of  simulator  time,  and  were  reimbursed  for  their  travel  and  lodging 
expenses. 

Eight  pilots  were  male.  Pilots'  ages  ranged  from  20  to  70  years,  with  a  median  of  34.2  years. 
Total  flight  time  ranged  from  375  to  15000  hours,  with  a  median  of  875  hours.  Total 
instrument  time  ranged  from  50  to  1500  hours,  with  a  median  of  110  hours.  Ten,  four,  and 
one  pilot  had  general  aviation  experience  in  aircraft  categories  A,  B,  and  C,  respectively.  The 
pilot  with  15000  total  hours  and  1500  instrument  hours  had  military  flight  experience  in 
categories  B,  C,  and  D  aircraft,  and  corporate  flight  experience  in  category  A  aircraft.  One 
pilot  had  corporate  flight  experience  in  category  C  aircraft. 

The  mean  percentages  of  time  pilots  reported  using  Jeppesen  lAP  charts  and  NOS  lAP  charts 
were  42%  and  58%,  respectively.  Four  pilots  reported  a  preference  for  Jeppesen  lAP  charts, 
five  pilots  reported  a  preference  for  NOS  lAP  charts,  and  one  pilot  expressed  no  preference. 


2.2  APPARATUS 

Instrument  approach  procedure  flAPJ  charts  Each  of  the  approaches  were  depicted  in  four 
charting  formats,  as  shown  in  Table  1.  Each  approach  was  based  on  an  actual  approach  taken 
from  the  U.S.  Terminal  Procedures  charts  published  by  the  U.S.  National  Oceanic  and 
Atmospheric  Administration  (1994a,  1994b,  1994c,  1994d,  1994e,  1994f). 

Gardner  Mimicipal  Airport  (GDM)  was  selected  as  the  study  site  because  it  has  relatively  low 
amounts  of  traffic  during  the  week,  and  previous  studies  have  constructed  and  safely  flown 
prototype  Global  Positioning  System  (GPS)  approaches  at  GDM.  The  GPS  overlay  approaches 
were  constructed  because  it  was  not  possible  to  use  actual  VOR  or  NDB  approaches  for  this 
experiment.  It  would  have  been  desirable  to  have  had  pilots  fly  actual  VOR  and  NDB 
approaches.  However,  GDM  and  other  nearby  airports  did  not  have  enough  of  these 
established  instrument  approaches.  Even  if  enough  of  these  approaches  were  available,  there 
would  have  been  an  unacceptably  high  level  of  overlap  for  frequencies  used  for 
communication  and  for  tuning  navigation  instruments,  and  geographic  features.  Additionally, 
local  pilots  would  have  memorized  many  of  these  features,  flight  paths  and  frequencies.  As  a 
result,  non-precision  approaches  from  around  the  United  States  were  selected  for  the  study. 
The  objective  of  the  selection  was  primarily  to  obtain  a  range  of  information  density  in  the 
group  of  charts. 
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Table  1. 

Instrument  Approach  Procedure  Charts  Used  in  This  Experiment 


NOS 

Prototype 

Rwy  18 

Rwy  36 

Rwy  18 

Rwy  36 

AVP 

BTV 

COS 

DFW 

FDK 

MFR 

RNO 

SAT 

For  each  chart  taken  from  the  U.S.  Terminal  Procedures  charts,  the  distance  and  bearing  of 
each  fix  (initial,  final,  and  other  fixes)  from  that  particular  runway  end  were  calculated.  The 
geometric  relationship  between  those  fixes  was  the  flight  path  of  that  approach  procedure.  The 
latitude  and  longitude  of  the  runway  ends  at  GDM  were  known.  To  replicate  the  geometry  of 
an  approach,  the  GDM  runway  end  was  used  as  a  starting  point  and  the  distance  and  bearing 
of  each  fix  was  plotted  from  that  point. 

The  charted  procedures  were  actually  flown  in  this  experiment,  and  because  of  prevailing 
winds,  some  pilots  flew  some  or  all  of  their  approaches  to  runway  1 8  at  GDM,  and  some  or 
all  of  their  approaches  to  runway  36.  Each  pilot  was  assigned  his/her  specific  order  of 
^proaches  according  to  a  counterbalancing  schedule.  Because  of  the  method  used  to 
construct  the  GPS  approaches,  the  landing  direction  was  irrelevant  to  the  counterbalancing 
schedule.  Each  GPS  procedure  constructed  for  an  approach  to  one  runway  end  (e.g.,  AVP 
Rwy  18)  was  rotated  approximately  180°  to  construct  the  GPS  procedure  for  the  approach  to 
the  opposite  nmway  end  (e.g.,  AVP  Rwy  36).  Therefore,  for  any  approach  (AVP,  BTV,  etc.) 
the  geometry  of  the  procedures  to  both  runway  ends  were  identical. 

The  charts  were  drawn  using  Adobe  Illustrator  5.5  installed  on  an  Apple  Macintosh  Ilci 
computer,  and  were  printed  with  a  Hewlett-Packard  Laser  Jet  Ulsi  with  400  dots  per  inch 
emulated  resolution. 

Frasca  242  flight  simulator  This  twin-engine  instrument  flight  simulator  is  a  fixed-base 
training  device  which  has  been  reconfigured  and  instrumented  for  data  collection.  There  is  no 
outside  visual  display.  The  specific  aircraft  model  used  for  this  study  was  designed  to 
approximate  the  flight  characteristics  of  a  Beechcraft  Baron  BE55.  A  depiction  of  the 
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instrument  panel  layout  is  given  in  Figure  4.  Both  in  the  simulator  and  in  the  airplane,  pilots 
chose  between  a  Thunderhead  training  visor  (instrument  hood)  or  Foggles. 

Simulated  GPS  receiver  A  simulated  GPS  receiver  in  the  Frasca  displayed  navigation 
information  on  the  liquid  crystal  display  (LCD)  panel.  The  display  showed  the  previous 
waypoint,  the  waypoint  being  flown  to,  the  distance  to  that  waypoint,  and  the  bearing  to  that 
waypoint  (see  Figure  5).  The  simulated  receiver  also  drove  the  course  deviation  indicator 
(CDI)  on  the  horizontal  situation  indicator  (HSI).  Needle  sensitivity  was  set  to  3/10  nm  full 
scale. 

Beechcraft  Baron  BE55  This  twin-engine  four-seat  aircraft  was  selected  because  in  single 
pilot  operations  it  creates  a  relatively  high  level  of  pilot  workload  during  terminal  area 
procedures  (Huntley,  Turner,  and  Palmer,  1993).  Figure  6  depicts  the  instrument  panel  layout. 
The  aircraft  was  based  at  a  fixed  base  operator's  facilities  at  Laurence  G.  Hanscom  Field 
(BED).  As  a  safety  precaution,  both  the  pilot  and  the  safety  pilot  had  a  complete  set  of 
current  NOS  lAP  approach  charts  available  to  them  at  all  times. 

Garmin  155  TSO  GPS  receiver  Course  guidance  in  the  Baron  was  provided  by  a  Garmin 
GPS  receiver  mounted  in  the  radio  stack  in  front  of  the  safety  pilot.  The  receiver's  display 
showed  the  previous  waypoint,  the  waypoint  being  flown  to,  the  distance  to  that  waypoint,  a 
CDI,  the  ground  speed,  the  bearing  to  that  waypoint,  and  the  aircraft  track  over  the  ground 
(see  Figure  7).  The  receiver  also  drove  the  CDI  on  the  HSI.  Needle  sensitivity  was  set  to  3/10 
nm  full  scale. 

Liquid  Crystal  Light  Shutter  One  6x9  inch  Taliq  Corporation  liquid  crystal  light  shutter  was 
mounted  between  two  plates  of  .15  inch  thick  Lexan  plexiglass  in  a  6.4  x  9.5  x  0.4  inch 
aluminum  frame  (see  Figure  8).  The  frame  was  open  on  one  side  to  allow  insertion  and 
extraction  of  the  charts.  This  .05  inch  thick  light  shutter  was  a  polymer  encapsulated  numatic 
curvilinear  aligned  phase  device  whose  liquid  crystal  molecules  were  not  aligned  in  parallel. 
This  non-parallel  alignment  resulted  in  the  scattering  of  light  which  rendered  the  device 
translucent  but  not  clear.  When  a  90  volt  6.67  mA  AC  electric  field  was  applied  across  the 
device,  the  molecules  aligned  and  the  shutter  cleared  in  10  msec.  When  the  electric  field  was 
removed,  the  shutter  returned  to  the  masked  condition  in  30  msec. 

All  charts  were  mounted  on  a  6  x  8.5  inch  piece  of  cardboard.  The  shutter  unit  was  sized  to 
accept  this  cardboard  moimting  when  the  safety  pilot  dropped  it  into  the  housing  behind  the 
shutter.  Charts  were  held  in  place  with  a  Velcro  strap  placed  across  the  housing's  open  side. 

The  shutter  was  in  its  masked  state  until  activated  to  its  clear  state  by  the  pilot.  A  push-to-talk 
switch  was  attached  to  the  pilot's  yoke  with  Velcro  straps.  This  toggle  switch  was  used  to 
operate  the  shutter.  The  switch  was  spring  loaded  and  set  toggled  to  the  "off  position,  which 
was  the  "masked"  state  for  the  shutter.  The  toggle  had  be  activated  and  held  active  to  clear 
the  shutter.  When  the  toggle  was  released,  the  shutter  returned  to  its  masked  state.  A  10  Hertz 
sampling  rate  was  used  by  the  Gateway  2000  Handbook  486  laptop  computer  to  detect 
whether  the  switch  was  activated. 
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Figure  4.  Frasca  242  Flight  Simulator  Cockpit  Layout 


FIPA  ->  KATY 
DTW  4  nm 
BRG  003  deg 


Figure  5.  Simulated  GPS  receiver  display  showed  the  previous  waypoint  (FIPA), 
the  waypoint  being  flown  to  (KATY),  the  distance  to  that  waypoint 
(4  nm),  and  the  bearing  to  that  waypoint  (003°) 

Laptop  computer  A  Gateway  2000  Handbook  486  computer  controlled  both  the  data 
collection  software  and  hardware  interface  for  the  light  shutter.  The  experimenter's  graphic 
interface  was  written  in  Visual  Basic. 


2.3  PROCEDURE 

Table  2  delineates  the  series  of  activities  each  pilot  stepped  through.  As  shown  in  this  table, 
pilots  participated  for  two  days.  The  first  day  consisted  of  training  in  the  Frasca  242 
simulator.  The  purpose  of  this  first  day  was  to  give  the  pilots  experience  in  using  the 
prototype  chart  format,  flying  with  guidance  from  a  GPS  receiver^  and  having  someone  ask 
them  a  series  of  questions  while  flying  an  approach.  Data  collection  during  actual  flight 
occurred  on  the  second  day.  Both  days  were  composed  of  a  morning  session  and  an  afternoon 
session.  In  each  session,  pilots  were  exposed  to  only  one  chart  format  (NOS  or  prototype). 
Pilots  were  randomly  assigned  to  be  trained  first  on  either  the  NOS  or  prototype  format 
charts.  All  pilots  received  training  for  and  flew  with  both  chart  formats. 

The  simulator  day  preceded  the  airplane  day  by  no  more  than  7  days.  However,  on  the 
airplane  day,  pilots  were  not  expected  to  recall  all  the  experimental  procedures  and 
differences  between  the  chart  formats.  Therefore,  the  experimenter  refamiliarized  pilots  with 
the  procedures  and  formats  before  pilots  began  the  airplane  day's  practice  trials.  Pilots  were 
told  that  each  session  (morning  and  afternoon)  consisted  of  a  practice  trial  followed  by  four 
data  collection  trials. 

Simulator  Dav:  Questionnaires  and  Chart  Format  Training  Pilots  were  seated  at  a  table  and 
read  an  informed  consent  form  summarizing  the  purpose  and  general  procedures  of  the 
experiment  (see  Appendix  A).  They  then  completed  a  questionnaire  concerning  their  flight 
experience  and  preferences  for  lAP  chart  manufacturers  (see  Appendix  B).  Pilots  then  read  a 
description  of  the  flight  procedures  (see  Appendix  C),  and  the  experimenter  answered  any 
procedural  questions. 
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Figure  6.  Beechcraft  Baron  BE55  Cockpit  Layout 


FIPA  ->  KATY 

gs: 

120S 

dis  4.00"m 

brg: 

003° 

+  +  +  +  -i-t  +  -t-  +  -l--l- 

trk 

003° 

Figure  7.  Garmin  GPS  155  receiver  display  showed  the  previous  waypoint 
(FIPA),  the  waypoint  being  flown  to  (KATY),  the  distance  to  that 
waypoint  (4  nm),  a  GDI,  the  ground  speed  (120  kt),  the  bearing  to 
that  waypoint  (003°),  and  the  aircraft  track  over  the  ground  (003°) 


Then  the  pilot  was  seated  in  the  left  seat  of  the  simulator,  and  had  an  opportunity  to  become 
familiar  with  the  instrument  suite.  Unlike  the  airplane  session,  no  safety  pilot  was  present  in 
the  simulator.  When  the  experimenter  was  satisfied  that  the  pilot  understood  the 
instrumentation  and  avionics  layout,  the  pilot  was  allowed  to  fly  for  15  minutes,  and  was 
instructed  to  fly  whatever  maneuvers  he/she  wished  to  get  familiar  with  the  handling  and 
flight  characteristics  of  the  simulator.  During  this  time  the  pilot  was  encouraged  to  ask 
questions.  At  the  conclusion  of  this  15  minute  period,  the  experimenter  brought  the  pilot  out 
of  the  simulator  and  back  to  the  table  for  chart  format  training. 

Simulator  Dav  Training  (Airplane  Day  RefamiliarizationV  NOS  Format  Charts  The 
experimenter  began  by  showing  pilots  an  NOS  format  lAP  chart  of  a  GPS  overlay  VOR 
approach  to  runway  25R  at  Los  Angeles  International  Airport  (LAX).  Both  training  charts  are 
shown  in  Appendix  D.  The  experimenter  reviewed  the  information  elements  (e.g., 
communication  frequencies,  missed  approach  instructions,  etc.)  and  their  locations  throughout 
the  chart,  referring  to  the  guide  shown  in  Appendix  E  to  ensure  that  each  section  of  the  chart 
was  reviewed.  Pilots  were  allowed  as  much  time  as  they  wished  to  review  the  chart  by 
themselves  and  were  encouraged  to  ask  questions.  After  subjects  indicated  that  they 
understood  the  format,  the  simulator  or  airplane  activities  began. 

Simulator  Dav  Training  (Airplane  Dav  RefamiliarizationL  Prototype  Format  Charts  Unlike 
any  of  the  other  information  on  the  prototype  versions  of  the  lAP  charts,  iconic  missed 
approach  instructions  were  a  new  concept  for  the  pilots.  Training  in  interpreting  iconic  missed 
approach  instructions  began  with  pilots  studying  the  examples  shown  in  Appendix  F.  These 
examples  were  composed  of  similar  kinds  of  instructions  pilots  would  encounter  during  the 
experiment.  Pilots  had  as  much  time  as  they  wished  to  study  the  examples  and  were 
encouraged  to  ask  questions.  When  they  completed  studying  the  examples,  the  experimenter 
gave  pilots  the  self  test  shown  in  Appendix  G.  After  completing  the  self-test  on  their  own,  the 
experimenter  reviewed  the  items  in  a  pseudorandom  order,  and  any  errors  were  corrected  by 
the  experimenter.  None  of  the  pilots  misinterpreted  the  icons.  Pilots  were  required  to 
demonstrate  100%  accuracy  in  order  to  proceed  to  the  prototype  format  training.  All  pilots 
met  this  criterion. 
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Figure  8. 


Physical  Dimensions  of  the  Liquid  Crystal  Light  Shutter 


Table  2. 

Sequence  of  Activities  and  Hypothetical  Counterbalanced 
Order  of  Experimental  Conditions 


Day  1:  Training  in  the  simulator 
Informed  Consent  Form 

Flight  Experience  and  Chart  Usage  Questionnaire 
Description  of  flight  procedures 
Familiari2:ation  with  simulator  operation 
Training:  NOS  format  charts 
Practice  trial  with  NOS  format  chart 
Data  trial  with  NOS  format  chart 
Data  trial  with  NOS  format  chart 
Data  trial  with  NOS  format  chart 
Data  trial  with  NOS  format  chart 


missed  approach 
missed  approach 

land  -  subject  gets  out  of  sim  for  a  break 
missed  approach 
missed  approach 


Lunch 

Training:  prototype  format  charts 
Practice  trial  with  prototype  format  chart 
Data  trial  with  prototype  format  chart 
Data  trial  with  prototype  format  chart 
Data  trial  with  prototype  format  chart 
Data  trial  with  prototype  format  chart 


missed  approach 
missed  approach 

land  -  subject  gets  out  of  sim  for  a  break 
missed  approach 
missed  approach 
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Table  2  (cont.) 

Sequence  of  Activities  and  Hypothetical  Counterbalanced 
Order  of  Experimental  Conditions 


Day  2:  Airplane 

Description  of  flight  procedures 

Familiarization  with  Baron  BE  55 

Training  (refamiliarization):  NOS  format  charts 

Practice  trial  with  NOS  format  chart 

missed  approach 

Data  trial  with  NOS  format  chart 

missed  approach 

Data  trial  with  NOS  format  chart 

land  -  subject  gets  out  of  airplane  for  a  break 

Data  trial  with  NOS  format  chart 

missed  approach 

Data  trial  with  NOS  format  chart 

missed  approach 

Lunch 

Training  (refamiliarization):  prototype  format  charts 

Practice  trial  with  prototype  format  chart 

missed  approach 

Data  trial  with  prototype  format  chart 

missed  approach 

Data  trial  with  prototype  format  chart 

land  -  subject  gets  out  of  airplane  for  a  break 

Data  trial  with  prototype  format  chart 

missed  approach 

Data  trial  with  prototype  format  chart 

missed  approach 

The  experimenter  then  showed  pilots  the  prototype  version  of  the  GPS  overlay  VOR  approach 
to  runway  25R  at  LAX  shown  in  Appendix  D.  Following  the  guide  shown  in  Appendix  E,  the 
experimenter  verbally  explained  the  types  and  locations  of  information  elements  throughout 
the  chart.  Pilots  were  allowed  as  much  time  as  they  wished  to  review  the  charts  by 
themselves  and  were  encouraged  to  ask  questions. 

Airplane  or  Simulator  Days:  Flving  the  Charted  Approaches  GPS  approaches  can  be 
constructed  and  loaded  as  flight  plans  in  most  GPS  receivers'  databases.  If  a  pilot  wished  to 
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fly  a  particular  approach,  that  flight  plan  could  be  called  up  and  activated.  The  receiver  would 
then  provide  course  guidance  based  on  the  aircraft's  position  relative  to  the  waypoints 
specified  in  the  flight  plan.  Therefore,  the  number  of  GPS  approaches  possible  for  any  runway 
are  limited  only  by  the  receiver's  memory  capacity  and  regulations  imposed  on  the 
construction  of  terminal  instrument  procedures.  As  long  as  the  charts  depict  a  profile  for 
approaches  to  the  runway  at  GDM  (and  a  safety  pilot  is  present),  any  chart  used  to  approach 
and  land  at  GDM  can  appear  to  be  drawn  for  any  airport  with  a  runway  having  a  similar 
heading.  Since  it  was  possible  to  provide  actual  flight  guidance  when  pilots  flew  GPS 
approaches  which  were  created  specifically  for  the  study,  the  chart  formats  were  evaluated  as 
the  approaches  they  depicted  were  being  flown. 

By  having  pilots  fly  the  approaches,  the  ability  of  pilots  to  retrieve  information  from  the  chart 
formats  could  be  tested  in  the  operational  environment  in  which  the  charts  are  actually  used, 
under  realistic  levels  of  pilot  workload.  The  pilots  also  used  the  charts  to  obtain  heading, 
altitude,  MDA,  and  missed  approach  information  even  when  they  were  not  being  asked 
questions  by  the  experimenter. 

Simulator  Dav:  Preparation  for  Practice  Trials  When  the  chart  format  training  was 
completed,  the  experimenter  brought  the  pilot  back  to  the  left  seat  of  the  simulator.  Although 
the  simulator  had  no  outside  visual  display,  pilots  were  required  to  wear  their  choice  of 
foggles  or  an  instrument  hood  to  more  accurately  simulate  conditions  during  the  actual  flight 
portion  of  the  study.  Although  no  light  shutter  data  were  collected  in  the  simulator  session, 
pilots  were  instructed  to  "Fly  to  the  same  standards  you  trained  to  for  your  instrument 
practical  test.  Fly  each  approach  at  120  knots  ±10  knots,  and  maintain  the  altitudes  ±100 
feet." 

Pilots  were  informed  that  course  guidance  would  be  provided  by  a  simulated  GPS  receiver. 
The  simulated  receiver's  display  showed  the  waypoint  you  were  flying  to,  the  distance  to  that 
waypoint,  and  the  bearing  to  that  waypoint  (see  Figure  5).  This  display  was  presented  on  an 
LCD  panel  in  the  center  of  the  instrument  suite.  The  receiver  also  drove  the  course  deviation 
indicator  (CDI)  on  the  horizontal  situation  indicator  (HSI).  Needle  sensitivity  was  set  to  3/10 
nm  full  scale. 

Pilots  were  told  that  after  they  crossed  the  MAP,  they  were  then  to  disregard  the  GPS  course 
guidance  and  use  as  their  source  of  guidance  the  NAV  2  OBS  which  had  been  tuned  to  the 
GARDNER  VOR  (110.6).  In  every  non-landing  trial,  pilots  had  to  intercept  a  different  radial 
leading  to  the  missed  approach  holding  fix.  This  was  done  in  order  to  give  pilots  ample 
opportunity  to  gain  experience  in  using  the  iconic  missed  approach  instructions. 

Airplane  Dav:  Preparation  for  Practice  Trials  The  pilot  was  seated  at  a  table  and  read  a 
description  of  the  flight  procedures  (see  Appendix  C),  and  the  safety  pilot  and  experimenter 
answered  any  procedural  questions  the  pilot  had.  The  pilot  was  then  seated  in  the  left  seat  of 
the  airplane.  There  are  variations  in  instrumentation  and  avionics  among  different  models  of 
the  Beechcraft  Baron  BE-55.  Therefore,  the  pilot  was  verbally  briefed  by  the  safety  pilot  on 
the  aircraft's  equipment  before  the  engines  were  started.  Pilots  were  instructed  to  immediately 
surrender  control  of  the  aircraft  to  the  safety  pilot  at  any  time  the  safety  pilot  said  "I've  got 
it." 
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The  experimenter  then  introduced  the  light  shutter  to  the  pilot  and  referred  to  the  guide  shown 
in  Appendix  H  when  explaining  the  operation  of  the  shutter.  The  pilot  had  an  opportunity  to 
become  familiar  with  operating  the  shutter,  and  the  experimenter  emphasized  that  the  chart 
should  be  unmasked  only  when  the  pilot  wished  to  view  it,  and  to  keep  viewing  time  to  a 
minimum. 

The  pilot  and  safety  pilot  were  seated  in  the  left  and  right  seats,  respectively.  The  safety  pilot 
was  responsible  for  monitoring  the  safety  of  the  flight.  The  experimenter  was  seated  behind 
the  safety  pilot,  and  was  responsible  for  handing  charts  to  the  pilot  in  the  proper  order  from 
approach  to  approach,  asking  the  pilot  questions  and  entering  data  into  a  laptop  computer,  and 
extracting  and  inserting  charts  into  the  shutter. 

When  the  safety  pilot  was  satisfied  that  the  pilot  understood  the  differences  in  instrumentation 
and  avionics,  and  the  pilot  indicated  readiness  to  begin,  the  flight  to  GDM  began.  The  pilot 
took  off  from  Hanscom  Field  (BED).  After  departure  from  the  traffic  pattern  at  BED  the  pilot 
donned  his/her  choice  of  an  instrument  hood  or  foggles,  and  the  safety  pilot  gave  vectors  to 
GDM. 

Pilots  were  informed  that  course  guidance  would  be  provided  by  the  Garmin  GPS  155  TSO 
receiver  mounted  in  the  radio  stack  in  front  of  the  safety  pilot,  and  that  the  safety  pilot  would 
be  responsible  for  operating  the  receiver.  The  receiver's  display  showed  the  previous 
waypoint,  the  waypoint  being  flown  to,  the  distance  to  that  waypoint,  a  course  deviation 
indicator  (CDI),  the  ground  speed,  the  bearing  to  that  waypoint,  and  the  aircraft  track  over  the 
ground  (see  Figure  7).  The  receiver  also  drove  the  CDI  on  the  horizontal  situation  indicator 
(HSI).  Needle  sensitivity  was  set  to  3/10  nm  full  scale.  Pilots  were  told  that  after  they  crossed 
the  MAP,  they  were  then  to  disregard  the  GPS  course  guidance  and  use  as  their  source  of 
guidance  the  NAV  2  OBS  which  had  been  tuned  to  the  GARDNER  VOR  (110.6). 

Airplane  or  Simulator  Dav:  Practice  Trials  and  Data  Collection  The  procedures  for 
conducting  practice  trials  and  data  collection  were  the  same  for  the  simulator  and  airplane 
except  that  the  light  shutter  was  not  used  in  the  simulator.  The  pilot  had  an  opportunity  to  ask 
the  experimenter  questions  throughout  the  practice  trials.  The  chart  format  was  either  NOS  or 
prototype,  depending  upon  the  counterbalanced  order  for  that  pilot. 

To  begin  the  first  practice  trial,  the  experimenter  inserted  a  chart  into  the  shutter.  All  switch 
activations  were  recorded  by  the  laptop  computer  software,  and  any  activations  during  this 
process  were  recognized  as  calibration  checks  during  post-flight  data  analyses.  These  analyses 
employed  a  set  of  logic  operations  on  the  time  stamped  data  to  make  accurate  determinations 
as  to  which  toggle  activations  were  in  response  to  a  question,  which  activations  were  due  to 
the  pilot  independently  checking  the  chart  for  his/her  own  reasons,  and  which  activations 
should  be  totally  ignored  such  as  calibration  checks. 

At  this  point  in  the  approach,  the  safety  pilot  was  giving  vectors  to  the  pilot.  The  safety  pilot 
then  informed  the  pilot  and  the  experimenter  that  "I  am  activating  the  approach,"  and 
activated  the  appropriate  flight  plan  (approach  procedure)  in  the  GPS  receiver.  That  approach 
would  have  been  the  same  as  the  approach  which  was  depicted  on  the  pilot's  chart.  When  an 
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approach  was  activated  in  the  GPS  receiver,  the  unit  provided  the  pilot  with  distance  and 
bearing  from  the  aircraft  to  the  first  leg  of  the  approach. 

The  experimenter  then  asked  the  pilot  if  he/she  was  ready  to  begin  their  review  of  the 
approach.  If  the  pilot  indicated  that  he/she  was  ready,  the  experimenter  would  hand  the  shutter 
to  the  pilot,  as  shown  in  Figure  9.  When  the  experimenter  gave  the  shutter  to  the  pilot,  the 
experimenter  pressed  the  recording  function  key  (as  illustrated  in  Figure  10),  and  instructed 
the  pilot  not  to  look  at  the  chart  until  he/she  was  asked  the  first  (quick  reference)  question. 

(To  begin  each  trial  during  the  simulator  day,  the  experimenter  would  place  the  chart  face 
down  on  the  right  seat,  and  tell  the  pilot  not  to  look  at  it  until  it  was  time  to  answer  the  quick 
reference  question.) 

The  experimenter  then  pressed  a  key  to  bring  the  quick  reference  question  and  correct  answer 
on  the  laptop  computer  screen.  This  key  press  defined  the  beginning  of  the  quick  reference 
question  period.  The  experimenter  then  asked  the  pilot  the  quick  reference  question  which 
was  always  the  same  -  "What  is  the  frequency  of  the  VOR  [or  NDB]  that  serves  this 
approach,  the  final  approach  course,  and  the  altitude  of  the  final  approach  fix?" 

The  pilot  toggled  "clear"  before  answering.  The  pilot  might  have  held  the  toggle  "clear"  or 
toggled  "clear"  and  release  ("mask")  several  times  while  looking  for  the  answer.  The  laptop 
computer  software  recorded  the  times  for  each  switch  activation  and  release  (two  time  stamps 
defined  each  look  at  the  chart).  The  pilot's  answer  was  recorded  as  correct  or  incorrect  by  the 
experimenter  pressing  the  appropriate  key.  That  key  press  defined  the  end  of  the  quick 
reference  question  period  and  the  beginning  of  the  review  period.  The  experimenter  then  told 
the  pilot  to  begin  his/her  review  of  the  approach.  The  pilot  reviewed  the  approach  the  way 
he/she  normally  would,  without  interruption  from  the  safety  pilot  or  experimenter. 

The  pilot  told  the  experimenter  when  he/she  had  completed  the  review.  The  experimenter  then 
asked  the  pilot  "Are  you  ready  to  begin  the  question  and  answer  session?"  If  the  pilot 
indicated  that  he/she  was  ready,  the  experimenter  pressed  another  key  which  displayed  the 
next  question  with  its  accompanying  correct  answer.  The  laptop's  software  randomly  chose  the 
next  question  from  the  16  questions  shown  in  Table  3.  This  key  press  defined  the  end  of  the 
review  period. 

The  pilot  might  or  might  not  have  toggled  "clear"  before  answering.  If  the  pilot  remembered 
the  answer  from  the  review,  he/she  might  have  recited  it  from  memory.  The  pilot's  answer 
was  recorded  as  correct  or  incorrect  by  the  experimenter  pressing  the  appropriate  key.  That 
key  press  defined  the  end  of  the  question  2  period.  The  experimenter  then  pressed  a  key  to 
ask  for  the  next  question.  Each  question  addressed  an  information  element  whose  appearance 
and/or  formatting  had  been  changed  on  the  prototype  chart. 

This  procedure  was  repeated  for  each  of  the  remaining  questions.  While  enroute  to  the  lAF, 
eight  questions  were  asked;  from  the  lAF  to  the  FAF,  five  questions;  and  from  the  FAF  to  the 
missed  approach  point  (MAP),  four  questions  were  asked. 
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Transition  from  VOR-driven  OBS 

to  safety  pilot’s  vectors  - ► 

for  course  guidance  yit 


End  data  collection 
on  missed  approach  trials 


Pilot  returns  shutter 
to  experimenter 

Experimenter  extracts  old 
chart  and  inserts  fresh  chart 


Transition  from - ^ 

GPS-driven  GDI  to 
VOR-driven  OBS  for 
course  guidance 


End  data  collection 
on  landing  trials 


_  Experimenter  gives  shutter 

to  pilot 

Quick  reference  question 
Pilot  begins  review  of  approach 


8) - {1 


Transition  from  safety  pilot's  vectors 
to  GPS-driven  GDI  for  course  guidence 


Pilot  ends  review  of  approach 


Figure  9.  Sequence  of  Pilot,  Safety  Pilot,  and  Experimenter  Activities 
During  an  Approach  Trial 
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Figure  10.  Illustration  of  How  Search  Time  and  Accuracy  Data  Were  Obtained 


Table  3. 

Approach  Test  Questions 


What  is  the  frequency  of  the  [VOR  or  NDB]*  that  serves  this  approach? 

What  is  the  final  approach  course  heading? 

What  is  the  correct  altitude  at  the  final  approach  fix? 

What  is  the  touchdown  zone  elevation? 

What  is  the  airport  elevation? 

(A  question  on  the  notes  specific  to  that  chart.) 

What  is  the  approach  lighting  configuration  type? 

In  the  missed  approach  procedures,  what  is  the  [first  or  second  heading  or  first 
or  second  altitude]? 

What  is  the  name  of  the  missed  approach  holding  fix? 

What  is  the  [ATIS  or  AWOS-3]  frequency? 

What  is  the  [approach  or  UNICOM]  frequency? 

What  is  the  tower  frequency? 

What  is  the  [ground  or  clearance  delivery]  frequency? 

Approaching  the  airport  from  the  [N,  NE,  E,  SE,  S,  SW,  W,  NW]  what  is  the  MSA? 
Which  identifier  and  type  of  facility  is  the  MSA  circle  based  on? 

What  is  the  minimum  descent  altitude? 

*  One  of  the  items  shown  in  brackets  was  chosen  for  any  one  chart. 


Pilots  were  instructed  to  be  prepared  to  land  or  fly  the  missed  approach  procedure,  based  on 
the  safety  pilot's  directions.  If  they  were  not  specifically  told  to  land  by  the  safety  pilot 
immediately  prior  to  reaching  the  MAP,  they  would  fly  the  missed  approach  procedure  shown 
on  the  chart.  If  the  pilot  was  instructed  to  land,  the  experimenter  pressed  the  stop  recording 
key  when  the  safety  pilot  told  the  pilot  to  land.  This  key  press  ended  data  recording  and 
marked  the  end  of  the  trial.  At  that  point,  the  pilot  passed  the  shutter  back  to  the  experimenter 
as  a  safety  precaution.  If  the  pilot  was  not  instructed  to  land,  the  experimenter  continued  to 
record  data  until  the  aircraft  intercepted  the  radial  enroute  to  the  missed  approach  holding  fix. 


24 


The  first  practice  trial  was  followed  by  two  data  collection  trials.  The  pilot  was  told  to  land 
during  the  second  data  trial,  and  had  a  10  minute  break  during  which  he/she  was  required  to 
step  out  of  the  airplane  (or  simulator).  Two  more  data  collection  trials  were  then  flown  before 
lunch.  Immediately  following  lunch,  the  experimenter  trained/refamiliarized  the  pilot  on  the 
chart  format  which  was  not  flown  in  the  morning.  In  the  example  shown  in  Table  2,  that  chart 
format  would  have  been  the  prototype  format.  The  training  was  completed  according  to  the 
procedures  described  previously.  After  successfully  completing  training,  the  same  sequence  of 
one  practice  and  four  data  trials  was  conducted.  For  each  subject,  the  approaches  that  were 
used  for  the  data  collection  trials  in  the  morning  session  were  not  used  in  the  afternoon  data 
collection  trials.  For  each  subject,  the  order  of  exposure  to  all  independent  variables  was 
identical  between  the  simulator  and  airplane  sessions.  Combinations  such  as  Moming/NOS 
and  Moming/prototype  were  counterbalanced  between  subjects. 

Airplane  Dav:  Subjective  Assessments  After  returning  to  BED,  the  experimenter  and  pilot 
met  in  a  conference  room  at  the  fixed  base  operator.  The  experimenter  used  the  LAX  charts 
shown  during  training  to  conduct  the  subjective  assessments.  After  handing  the  NOS  LAX 
chart  and  the  prototype  LAX  chart  to  the  pilot,  the  experimenter  asked  "Which  chart  did  you 
prefer  for  answering  the  first  question  (frequency  of  the  VOR,  approach  course,  altitude  at  the 
final  approach  fix)  -  the  NOS,  the  prototype,  or  no  preference?"  The  experimenter  asked  for 
the  reasons  why  one  format  was  preferred  over  the  other,  or  why  there  was  no  preference,  and 
recorded  those  comments.  The  experimenter  then  asked  "Which  chart  did  you  prefer  for 
reviewing  the  approach  -  the  NOS,  the  prototype,  or  no  preference?"  The  next  question  was 
"Which  chart  did  you  prefer  for  executing  the  approach  -  the  NOS,  the  prototype,  or  no 
preference?"  The  experimenter  then  asked  "Which  chart  did  you  prefer  for  executing  the 
missed  approach  procedures  -  the  NOS,  the  prototype,  or  no  preference?"  Finally,  the 
experimenter  asked  "What  features  of  the  prototype  chart  did  you  not  use?" 


2.4  PERFORMANCE  MEASURES 

The  objective  of  the  prototype  format  was  to  increase  pilots'  speed  and  accuracy  of 
information  retrieval  from  approach  charts.  Therefore,  the  performance  measures  focused  on 
these  dependent  variables.  Search  time  was  obtained  by  measuring  the  elapsed  time  between  a 
depression  of  the  shutter  toggle  followed  by  a  release.  The  accuracy  of  answers  to  each  of  the 
17  questions  was  measured  as  either  correct  (1)  or  incorrect  (0). 

Determining  Search  Time  for  Each  Question  As  shown  in  Figure  10,  the  experimenter 
pressed  a  key  to  display  a  question  and  its  answer  on  the  laptop  computer  screen.  This  key 
press  defined  the  beginning  of  that  question's  period.  Any  toggle  clear  which  occurred  after 
the  experimenter  pressed  that  key  and  before  pressing  the  key  to  record  the  accuracy  of  the 
pilot's  response  was  recorded  as  being  in  response  to  that  question.  The  pilot  was  not  required 
to  toggle  "clear"  before  answering,  since  if  he/she  remembered  the  answer  from  a  previous 
look  they  would  not  need  to  look  again  for  that  answer.  The  experimenter's  key  press 
denoting  accuracy  defined  the  end  of  that  question's  time  period. 

Determining  Search  Time  for  Reviewing  the  Approach  Pilots  were  allowed  to  clear  the 
shutter  at  any  time.  A  question  was  not  required.  The  pilot  might  have  decided  that  it  was 
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necessary  or  desirable  to  examine  the  chart  at  that  particular  moment.  Although  these  shutter 
activations  could  occur  all  along  the  approach,  only  those  occurring  during  the  period  when 
the  pilot  was  reviewing  the  approach  were  of  interest. 

As  shown  in  Figure  9,  the  entire  review  session  which  occurred  prior  to  the  lAF  consisted  of 
one  or  a  series  of  these  shutter  activations.  The  search  time  for  the  review  period  was  defined 
as  the  total  elapsed  time  the  shutter  was  clear  during  this  period.  This  period  was  defined  in 
the  data  as  beginning  with  the  experimenter's  key  press  to  record  the  accuracy  of  the  answer 
to  the  quick  reference  question,  and  ending  with  the  experimenter's  key  press  to  call  for  the 
second  question.  No  questions  were  asked  during  the  pilot's  review  of  the  approach. 

Subjective  Assessments  Both  quantitative  and  qualitative  data  were  obtained  in  this  portion 
of  the  study.  For  the  quick  reference  question  and  for  each  approach  phase  (review,  execution, 
and  missed  approach),  whether  the  pilot  preferred  a  chart  (1)  or  had  no  preference  (0)  was 
recorded,  as  well  as  the  reasons  for  that  response. 

For  the  open-ended  questions  regarding  the  features  of  the  prototype  chart  which  pilots  did 
not  use  and  if  they  had  any  other  comments,  the  comments  were  recorded  and  frequency 
counts  of  the  comments'  topics  were  computed. 

Optimizing  Data  Collection  Accuracy:  Standbys  and  Lock-outs  Experience  gained  from 
pretesting  the  experimental  procedures  showed  that  sometimes  pilots  need  to  access  the  chart 
for  information  that  has  nothing  to  do  with  the  question  currently  being  asked.  For  example, 
the  experimenter  might  press  a  key  to  bring  up  the  next  question,  which  might  be  "What  is 
the  approach  frequency?",  while  at  that  particular  moment  the  pilot  might  be  preoccupied  with 
trying  to  find  the  minimum  descent  altitude.  The  result  would  be  that  toggle 
clears  for  the  minimum  descent  altitude  would  get  combined  with  the  toggle  clears  for  the 
approach  frequency  and  artificially  inflate  the  search  time  needed  to  answer  the  approach 
frequency  question. 

Although  randomizing  the  order  in  which  questions  were  asked  rendered  this  source  of  noise 
unsystematic,  the  sensitivity  of  the  performance  measure  was  reduced.  This  problem  was 
solved  by  creating  a  standby  option  for  the  pilot  and  experimenter.  The  pilot  was  instructed  to 
say  "Standby."  if  the  experimenter  asked  a  question  and  the  pilot  was  preoccupied  with 
another  chart  item  or  any  other  flying  task.  With  this  procedure  in  place,  if  the  experimenter 
pressed  a  key  to  bring  up  the  next  question  such  as  "What  is  the  approach  frequency?"  while 
at  that  particular  moment  the  pilot  was  preoccupied  with  trying  to  find  the  minimum  descent 
altitude,  the  pilot  would  simply  have  said  "Standby."  and  the  experimenter  would  have 
recorded  a  comment  in  the  data  base  which  would  be  found  during  post-flight  data  analyses. 
The  toggles  would  then  be  corrected.  When  the  pilot  was  ready  to  resume,  he/she  would  have 
said  "Ready."  or  asked  the  experimenter  to  repeat  the  question.  Again,  the  experimenter  would 
enter  another  comment  in  the  data  file  so  that  the  upcoming  toggles  would  be  correctly 
attributed.  The  experimenter  would  then  repeat  the  question. 

Another  safeguard  to  improve  the  accuracy  of  determining  the  cause  of  toggles  was  controlled 
by  the  laptop  computer's  software.  When  the  pilot  toggled  the  shutter  clear,  the  experimenter 
was  locked  out  from  pressing  a  key  to  display  the  next  question.  As  soon  as  the  pilot  released 
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the  button  and  the  shutter  returned  to  its  default  masked  state,  the  experimenter  was  again 
able  to  display  the  next  question  screen  with  a  key  press.  Pilots  were  never  locked  out  from 
clearing  the  shutter,  regardless  of  what  the  experimenter  was  doing. 


2.5  EXPERIMENTAL  DESIGN 

The  main  effect  of  chart  format  (NOS  or  prototype)  was  the  main  effect  of  primary  interest. 
Preliminary  data  analyses  uncovered  another  main  effect,  which  was  the  order  in  which  pilots 
flew  the  chart  formats.  This  main  effect  for  format  order  had  two  conditions;  NOS-then- 
prototype,  and  prototype-then-NOS. 

Table  4  shows  the  experimental  design.  The  within-subjects  independent  variable  was  chart 
format.  The  between-subjects  independent  variable  was  format  order.  A  series  of  t  tests  (Kirk, 
1982)  were  conducted  on  the  search  time  and  accuracy  data  for  the  two  main  effects  and  their 
interaction. 


Table  4. 

Experimental  Design 


Prototype  then  NOS  format  Order 

Prototype 

NOS 

Pilot, 

Pilotj 

Pilots 

Pilotj 

Pilotg 

NOS  then  Prototype  Format  Order 

NOS 

Prototype 

Pilotj 

PiloL 

Pilots 

Pilotg 

Pilot, 0 
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3.  RESULTS 


3.1  DATA  ANALYSIS  PROCEDURE 

A  series  of  t  tests  (Kirk,  1982)  were  conducted  on  the  search  time  and  accuracy  data.  Two 
main  effects  and  their  interaction  were  tested  for  statistical  significance.  The  main  effect  of 
chart  format  (NOS  or  prototype)  was  the  main  effect  of  primary  interest.  Preliminary  data 
analyses  uncovered  another  main  effect,  which  was  the  order  in  which  pilots  flew  the  chart 
formats.  This  main  effect  for  format  order  had  two  conditions:  NOS-then-prototype,  and 
prototype-then-NOS.  The  nature  of  the  format  order  effect  is  addressed  in  the  Discussion 
section  of  this  report. 

The  first  t  test  conducted  for  any  dependent  variable  (e.g.,  the  mean  search  time  required  to 
answer  the  quick  reference  question)  was  conducted  on  the  interaction  of  chart  format  and 
format  order.  Within  each  format  order,  a  score  was  obtained  by  computing  the  mean 
difference  between  the  search  time  for  that  variable  for  NOS  charts  and  the  search  time  for 
that  variable  for  prototype  charts.  The  t  test  then  was  used  to  determine  if  the  difference  score 
for  one  format  order  differed  significantly  (p  <  .05)  from  the  difference  score  for  the  other 
format  order. 

If  the  interaction  were  significant,  then  the  effect  for  chart  format  would  have  been  shown  to 
be  dependent  upon  the  format  order  condition.  In  other  words,  in  order  to  analyze  the  effect 
of  the  chart  format,  the  order  in  which  the  formats  were  flown  had  to  be  taken  into  account. 
Therefore,  if  the  interaction  were  significant,  the  main  effect  for  chart  type  was  tested  within 
each  format  order  condition. 

If  the  interaction  were  not  significant  (p  >  .05),  then  the  assumption  that  format  order  did  not 
interact  with  chart  type  could  not  be  rejected.  Subsequently,  the  data  were  combined  across 
format  order  conditions,  and  a  score  was  obtained  by  computing  the  mean  difference  between 
the  search  time  for  that  variable  for  NOS  charts  and  the  search  time  for  that  variable  for 
prototype  charts.  A  t  test  was  then  conducted  on  this  difference  score  to  analyze  the  main 
effect  for  chart  type. 

The  search  time  and  accuracy  results  are  reported  in  the  same  order  in  which  they  were 
obtained  during  an  approach.  These  performance  data  are  followed  by  a  summary  of  the 
pilots'  subjective  preferences  for  chart  formats. 


3.2  QUICK  REFERENCE  QUESTION 

Table  5  presents  the  mean  search  time  (MST)  required  to  answer  the  quick  reference  question 
for  both  chart  formats.  The  t  test  conducted  on  the  interaction  of  chart  format  and  format 
order  was  not  significant  (p  <  .217).  The  data  were  then  pooled  across  format  order  conditions 
and  the  results  of  the  t  test  conducted  on  MST  are  summarized  in  Table  6.  Pilots  answered 
the  quick  reference  question  with  a  significantly  (p  <  .007)  faster  MST  for  prototype  charts 


29 


Table  5. 

Mean  Search  Time  (Seconds)  Required  to  Answer:  "What  is  the  frequency  of  the 
[VOR  or  NDB]*  that  serves  this  approach,  the  final  approach  course,  and  the  altitude 
at  the  final  approach  fix?" 


Chart  Format 

NOS 

Prototype 

Mean 

13.25 

5.53 

9.39 

(8.47) 

(2.06) 

(5.06) 

Note:  Standard  deviations  are  given  in  parentheses. 

*One  of  the  items  shown  in  brackets  was  chosen  for  any  one  chart. 


Table  6. 

T-Test  of  the  Difference  Between  NOS  and  Prototype  Charts  for  Mean  Search  Time 
(Seconds)  Required  to  Answer:  "What  is  the  frequency  of  the  [VOR  or  NDB]*  that 
serves  this  approach,  the  final  approach  course,  and  the  altitude  at  the  final  approach 
fix?" 


Variable 

N 

Mean 

Std  Error 

T 

P  >  |T| 

Difference  in  Elapsed  Time 

10 

7.72 

2.23 

3.47 

0.007 

*One  of  the  items  shown  in  brackets  was  chosen  for  any  one  chart. 


than  for  NOS  charts.  As  depicted  in  Figure  11,  the  MST  for  the  prototype  format  was  more 
than  twice  as  fast  as  the  MST  for  the  NOS  format 

T  tests  were  also  conducted  on  the  accuracy  of  pilots'  answers  to  the  quick  reference  question. 
Neither  the  interaction  {p  <  .733)  nor  the  main  effect  for  chart  format  {p  <  .100)  were 
significant 
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NOS  Prototype 


Chart  Format 

Figure  11.  Mean  Search  Time  (Seconds)  Required  to  Answer:  "What  is  the  frequency  of 
the  [VOR  or  NDB]  that  serves  this  approach,  the  final  approach  course,  and  the 
altitude  at  the  final  approach  fix?" 


3.3  STUDYING  FOR  THE  APPROACH 

The  next  set  of  t  tests  were  conducted  on  the  MST  required  for  pilots  studying  for  the 
approach.  No  questions  were  asked  during  this  period.  Neither  the  interaction  nor  the  main 
effect  for  chart  format  were  significant  (p  <  .072  and  ;?  <  .548,  respectively). 


3.4  ANSWERING  ALL  QUESTIONS 

Table  7  presents  the  MST  needed  to  answer  all  16  questions  (shown  in  Table  3)  for  both 
chart  formats.  The  t  test  conducted  on  the  interaction  of  chart  format  and  format  order  was 
not  significant  {p  <  .064).  The  data  were  then  pooled  across  format  order  conditions  and  the 
results  of  the  t  test  conducted  on  MST  are  summarized  in  Table  8.  The  MST  required  by 
pilots  to  answer  questions  was  significantly  {p  <  .0001)  faster  with  prototype  charts  than  with 
NOS  charts.  As  depicted  in  Figure  12,  MST  for  the  prototype  format  was  approximately  nine 
seconds  faster  than  MST  for  the  NOS  format. 

This  analysis  was  followed  by  t  tests  conducted  on  the  accuracy  of  pilots'  answers  to  the 
questions.  Neither  the  interaction  (p  <  .191)  nor  the  main  effect  for  chart  format  ip  <  .483) 
were  significant. 
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Table  7. 

Mean  Search  Time  (Seconds)  Required  to  Answer:  Questions* 


Chart  Format 

NOS 

Prototype 

Mean 

43.58 

34.62 

39.10 

(16.09) 

(14.41) 

(15.13) 

Note:  Standard  deviations  are  given  in  parentheses. 

*These  data  do  not  include  the  mean 

search  time  to  answer 

the  quick  reference  question. 

Table  8. 

T-Test  of  the  Difference  Between  NOS  and  Prototype  Charts  for 
Mean  Search  Time  (Seconds)  Required  to  Answer;  Questions* 


Variable 

N 

Mean 

Std  Error 

T 

P  >  |T| 

Difference  in 

10 

8.96 

1.33 

6.71 

0.0001 

Elapsed  Time 

Note:  *These  data  do  not  include  the  mean  search  time  to  answer  the  quick  reference 
question. 


3.5  BRIEFING  STRIP  LAYOUT 

Twelve  of  the  16  questions  addressed  items  located  in  the  prototype's  briefing  strip.  Except 
for  the  radio  frequencies  and  notes,  the  information  in  the  briefing  strip  could  also  be  found 
distributed  throughout  the  prototype  chart.  In  the  NOS  charts,  all  of  these  items  were 
formatted  in  the  current  distributed  layout. 
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Figure  12.  Mean  Search  Time  (Seconds)  Required  to  Answer  Questions 


Table  9  presents  the  MST  to  answer  11  of  the  12  questions  for  both  chart  formats  in  both 
format  order  conditions.  (The  MST  required  to  answer  "In  the  missed  approach  procedures, 
what  is  the  [first  or  second  heading  or  first  or  second  altitude]?"  was  analyzed  separately  and 
is  discussed  below.)  The  results  of  the  t  test  conducted  on  the  interaction  of  chart  format  and 
format  order  was  significant  (p  <  .018)  as  shown  in  Table  10.  T  tests  were  then  conducted  on 
the  main  effect  of  chart  type  within  each  format  order  condition,  and  the  results  are  also 
summarized  in  Table  10.  In  both  the  NOS-then-prototype  format  order  condition  and  the 
prototype-then-NOS  format  order  condition,  pilots  answered  the  1 1  questions  significantly 
faster  for  prototype  charts  than  for  NOS  charts  {p  <  .0001  and/?  <  .047,  respectively).  The 
interaction  is  depicted  in  Figure  13.  The  magnitude  of  the  difference  between  mean  search 
times  for  NOS  and  prototype  charts  is  higher  for  the  NOS-then-prototype  format  order 
condition.  A  detailed  examination  of  this  interaction  is  provided  in  the  Discussion  section  of 
this  report. 

T  tests  were  then  conducted  on  the  accuracy  of  pilots'  answers  to  this  subset  of  questions. 
Neither  the  interaction  (p  <  .257)  nor  the  main  effect  for  chart  format  ip  <  .366)  were 
significant. 
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Table  9. 

Mean  Search  Time  (Seconds)  Required  to  Answer: 
Questions*  Regarding  Information  Included  in  the  Briefing  Strip 


Chart  Format 


NOS 

Prototype 

Format  Order 

Mean 

NOS  then  Prototype 

38.64 

27.12 

32.88 

(13.24) 

(12.15) 

(12.68) 

Prototype  then  NOS 

22.87 

17.40 

20.14 

(4.65) 

(2.19) 

(2.94) 

Mean 

30.75 

22.26 

(12.52) 

(9.69) 

Note:  Standard  deviations  are  given  in  parentheses. 

*These  data  do  not  include  the  mean  search  time  to  answer  the  quick  reference  question, 
nor  for  the  missed  approach  procedure  question. 


3.6  MISSED  APPROACH  INSTRUCTIONS  QUESTION 

Although  the  entire  missed  approach  instructions  were  given  in  text  format  in  both  NOS  and 
prototype  charts,  only  prototype  charts  used  icons  to  depict  the  "up  and  out"  portion  of  the 
missed  approach  instructions.  Table  1 1  shows  the  MST  required  by  pilots  to  answer  a 
question  regarding  this  initial  portion  of  the  missed  approach  instructions.  The  t  test 
conducted  on  the  interaction  of  chart  format  and  format  order  was  not  significant  {p  <  .202). 
The  data  were  then  pooled  across  format  order  conditions  and  the  results  of  the  t  test 
conducted  on  MST  are  summarized  in  Table  12.  Pilots  answered  this  question  with  a 
significantly  {p  <  .023)  faster  MST  for  prototype  charts  than  for  NOS  charts.  The  MSTs  for 
both  chart  types  are  depicted  in  Figure  14. 
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Table  10. 

T-Tests  of  Mean  Search  Time  (Seconds)  Required  to  Answer: 
Questions*  Regarding  Information  Included  in  the  Briefing  Strip 
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NOS  then  Prototype  Prototype  then  NOS 

Chart  Format  in  Both  Format  Orders 

Figure  13.  Mean  Search  Time  (Seconds)  Required  to  Answer  Questions 
Regarding  Information  Included  in  the  Briefing  Strip 

Table  11. 

Mean  Search  Time  (Seconds)  Required  to  Answer:  "In  the  missed  approach 
procedures,  what  is  the  [first  or  second  heading  or  first  or  second  altitude]*?" 


Chart  Format 

NOS 

Prototype 

Mean 

2.76 

1.98 

2.37 

(1.16) 

(0.60) 

(0.81) 

Note:  Standard  deviations  are  given  in  parentheses. 

*One  of  the  items  shown  in  brackets  was  chosen  for  any  one  chart. 
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Table  12. 

T-Test  of  the  Difference  Between  NOS  and  Prototype  Charts  for  Mean  Search  Time 
(Seconds)  Required  to  Answer:  ’*In  the  missed  approach  procedures,  what  is  the  [first 
or  second  heading  or  first  or  second  altitude]*?” 


Variable  N 

Mean 

Std  Error 

T 

P  >  |T1 

Difference  in  Elapsed  Time  10 

0.77 

0.28 

2.74 

0.023 

*One  of  the  items  shown  in  brackets  was  chosen  for  any  one  chart. 
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Figure  14.  Mean  Search  Time  (Seconds)  Required  to  Ajiswer:  ”In  the  missed 
approach  procedures,  what  is  the  [first  or  second  heading  or  first  or 
second  altitude]? 
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Pilots  made  no  errors  in  answering  the  missed  approach  instruction  question  while  using 
either  chart  type. 


3.7  PILOT  PREFERENCES  FOR  CHART  FORMATS 

The  experimenter  asked  pilots  whether  they  preferred  one  of  the  chart  formats,  or  had  no 
preference  for  accomplishing  the  following;  answering  the  quick  reference  question, 
reviewing  the  approach,  executing  the  approach,  and  executing  the  missed  approach.  The 
pilots'  responses  are  summarized  in  Table  13,  and  their  comments  are  provided  in  Appendix  I. 

The  NOS  chart  was  never  preferred  by  any  pilot  for  performing  any  of  the  tasks.  The 
prototype  chart  was  unanimously  preferred  for  answering  the  quick  reference  question  and  for 
executing  the  missed  approach  procedures. 


3.8  PILOT  COMMENTS 

The  experimenter  asked  pilots  to  list  the  features  of  the  prototype  chart  which  they  did  not 
use.  The  pilots'  responses  are  summarized  in  Table  14,  and  their  comments  are  also  provided 
in  Appendix  I.  Five  items  were  mentioned.  All  five  were  features  shown  in  the  briefing  strip 
which  are  also  shown  in  their  usual  location  on  current  NOS  charts. 

Finally,  the  experimenter  asked  pilots  if  they  had  any  other  comments.  All  of  the  pilots' 
responses  were  in  regard  to  the  prototype  chart.  Table  15  summarizes  their  comments  which 
are  provided  in  Appendix  I.  Again,  the  briefing  strip  was  the  pilots'  primary  subject  for 
comments. 


Table  13. 

Pilot  Preferences  for  Each  Chart  Format  for 
Quick  Reference  and  Each  Approach  Phase 


Chart  Format 

Approach  Phase 

NOS 

Prototype 

No  Preference 

Quick  Reference 

0 

10 

0 

Studying 

0 

8 

2 

Executing 

0 

6 

4 

Missed  Approach 

0 

10 

0 
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Table  14. 

Pilot  Responses  to  "What  Features  of  the  Prototype  Chart  Did  You  Not  Use?" 


Briefing  Strip  Feature  Not  Used 

Number  of  Pilots  Citing  That  Feature 

Airport  Elevation 

6 

Final  Approach  Fix  Information 

5 

Final  Approach  Course 

5 

Approach  Lighting  Sketch 

1 

Frequency  of  Primary  Navaid 

1 

Table  15. 

Pilot  Responses  to  "Do  You  Have  Any  Other  Comments?" 


Prototype  Chart  Feature  Positive  Comments 

Negative  Comments 

Briefing  Strip 

11 

2 

Highlighting  Text  with  Bolding  and  Boxing 

8 

0 

MSA  Circle 

6 

0 

Iconic  Missed  Approach  Instructions 

1 

0 
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4.  DISCUSSION 


4.1  MAJOR  FINDINGS 

The  objective  of  this  experiment  was  to  evaluate  the  effectiveness  of  the  prototype  lAP  chart 
design  in  transferring  information  to  pilots  during  actual  flight.  Four  specific  hypotheses  were 
tested,  and  each  one  predicted  that  pilots  would  retrieve  information  faster  and  more 
accurately  with  the  prototype  format  than  with  the  NOS  format.  Pilots  were  consistently  faster 
when  answering  questions  with  the  prototype  format.  This  significant  increase  in  speed  was 
obtained  at  no  cost  to  accuracy.  Pilots  were  as  accurate  in  answering  questions  using  the 
prototype  format  as  they  were  with  the  more  familiar  NOS  format. 

The  first  hypothesis  stated  that  the  formatting  of  the  briefing  strip  would  produce  faster  and 
more  accurate  retrieval  of  the  information  it  contained  than  the  formatting  of  that  information 
on  NOS  charts.  The  particular  analyses  that  provided  the  test  of  this  hypothesis  are  those  for 
the  quick  reference  question,  studying  for  the  approach,  and  the  briefing  strip  layout.  Data  for 
the  missed  approach  instruction  question  were  analyzed  separately  from  the  briefing  strip  data. 

The  quick  reference  question  asked  "What  is  the  frequency  of  the  [VOR  or  NDB]  that  serves 
this  approach,  the  final  approach  course,  and  the  altitude  at  the  final  approach  fix?".  When 
pilots  were  using  the  prototype  chart,  they  found  the  answer  to  this  question  more  than  twice 
as  fast  as  when  they  were  using  the  NOS  chart.  This  significant  difference  is  depicted  in 
Figure  1 1 .  The  reason  for  this  difference  may  be  seen  by  examining  the  charts  in  Appendix 
D.  The  first  three  boxes  in  the  top  line  of  the  prototype's  briefing  strip  give  this  information. 
In  order  to  answer  the  question  with  the  NOS  chart,  pilots  had  to  search  through  the  plan  and 
profile  views.  The  pilot  comments  in  Appendix  I  support  this  explanation.  Pilots  unanimously 
preferred  the  prototype  chart  for  answering  the  quick  reference  question  (see  Table  13). 

The  briefing  strip  formatting  was  also  expected  to  reduce  the  amount  of  time  pilots  spent 
when  they  were  studying  to  prepare  for  executing  the  approach.  Because  much  of  the  critical 
information  was  located  close  together,  pilots  were  expected  to  retrieve  the  information  more 
quickly.  However,  the  data  did  not  support  this  expectation.  No  significant  difference  for 
mean  search  time  (MST)  was  found  between  chart  formats.  One  explanation  for  not  detecting 
a  difference  between  the  chart  formats  is  that  the  pilots'  major  areas  of  interest  on  the  chart 
when  they  studied  the  ^proach  were  the  plan  view  in  particular,  as  well  as  the  profile  view. 
An  examination  of  the  chart  formats  reveals  few  differences  between  the  prototype  and  NOS 
charts  in  these  sections  of  the  charts.  Since  no  questions  were  asked  during  this  period,  pilots 
may  have  spent  the  majority  of  their  time  studying  chart  features  which  were  common  to  both 
chart  formats.  Although  no  statistically  significant  difference  was  found  for  MST  while 
studying,  eight  of  the  10  pilots  still  preferred  the  prototype  chart  for  studying  for  the 
approach,  as  shown  in  Table  13  and  Appendix  I. 

Different  results  were  found  when  pilots  were  asked  to  search  for  information  contained  in  the 
briefing  strip.  Although  there  was  a  significant  interaction  between  chart  format  and  the  order 
in  which  the  pilots  used  the  two  formats,  the  prototype  chart  produced  significantly  faster 
MST  in  both  format  order  conditions,  as  shown  in  Figure  13.  The  interaction  is  statistically 
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significant  because  the  magnitude  of  the  difference  between  MST  for  NOS  and  prototype 
charts  is  higher  for  the  NOS-then-prototype  format  order  condition.  Two  probable  causes 
combined  to  produce  the  interaction.  Both  causes  will  be  discussed  individually,  followed  by 
an  explanation  of  how  their  combination  resulted  in  the  interaction. 

The  first  cause  was  that  pilots  spent  a  substantial  portion  of  their  attention  on  learning  how  to 
perform  the  experimental  procedures  while  flying  an  unfamiliar  airplane.  Although  the  pilots 
were  trained  in  the  Frasca  flight  simulator  in  the  general  flight  procedures  and  experimental 
methods,  they  still  needed  to  learn  how  to  transfer  the  skills  they  learned  in  the  laboratory  to 
actual  flying  in  the  Baron.  None  of  the  pilots  had  flown  this  particular  airplane  before,  and  all 
of  the  pilots  were  having  their  first  experience  with  using  GPS  guidance  to  fly  approaches. 
Learning  to  perform  the  experimental  procedures  while  flying  an  unfamiliar  airplane 
decreased  the  pilots'  attention  resources  available  for  searching  for  information  on  the  chart. 
As  the  day  progressed,  pilots  became  more  and  more  familiar  with  both  how  the  airplane 
handled,  and  how  to  integrate  the  GPS  guidance  into  their  maneuvering.  As  pilots  completed 
their  learning,  the  attention  resources  available  for  searching  through  the  chart  increased. 
Therefore,  in  the  morning  session,  pilots  had  less  available  attention  for  searching  for 
information  to  answer  questions  than  in  the  afternoon  session. 

In  order  to  explain  the  interaction,  two  effects  still  need  to  be  explained.  These  effects  are 
best  expressed  as  questions.  The  first  is:  why  is  the  MST  for  the  NOS-then-prototype  format 
order  slower  than  the  MST  for  the  prototype-then-NOS  format  order?  The  second  question  is: 
why  would  the  MST  for  NOS  charts  flown  in  the  afternoon  be  faster  than  the  MST  for 
prototype  charts  flown  in  the  afternoon?  Assuming  the  pilot  groups  were  not  different  on 
some  critical  value  (a  review  of  pilot  characteristics  did  not  reveal  such  a  difference),  there 
must  be  something  else  going  on.  Consider  the  pilots  in  the  prototype-then-NOS  format  order 
condition.  Perhaps  it  was  so  easy  to  use  the  prototype  chart  in  the  morning  that  pilots  were 
able  to  learn  the  flying  and  GPS  navigation  tasks  very  well,  so  that  by  the  time  they  were 
answering  questions  with  the  NOS  charts  in  the  afternoon,  they  were  able  to  dedicate  an 
optimal  amount  of  attention  resources  to  search  for  the  answers  with  an  effective  search 
strategy.  In  the  morning  session,  when  pilots  had  less  attention  available  for  searching  for 
information  to  answer  questions,  the  prototype's  briefing  strip  was  very  helpful.  By  the  end  of 
the  morning  session,  they  had  already  learned  how  to  both  fly  the  airplane  and  use  GPS 
information.  Even  so,  these  pilots  would  still  be  expected  to  have  an  MST  that  was  lower  for 
the  prototype  than  for  the  NOS  format,  and  that  was  the  case. 

Now  consider  the  pilots  in  the  other  format  order  condition,  the  one  in  which  the  NOS  chart 
was  flown  in  the  morning.  These  pilots  may  have  had  such  a  difficult  time  searching  for 
answers  with  a  lesser  amount  of  available  attention,  that  they  never  really  learned  either 
effective  search  strategies  for  the  NOS  format,  nor  did  they  learn  enough  about  how  the 
airplane  handles  and  how  to  navigate  with  GPS  information  in  order  to  avoid  continuing  the 
learning  process  into  the  afternoon.  Although  the  MST  for  the  prototype  would  still  be 
expected  to  be  lower  than  the  MST  for  NOS,  these  pilots  would  be  expected  to  show 
relatively  high  MSTs  for  both  the  NOS  and  prototype  formats  compared  to  the  pilots  who 
flew  with  the  prototype  in  the  morning.  That  is  what  the  data  show. 
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These  causes  would  explain  the  difference  in  MSTs  between  the  format  order  conditions  (the 
NOS-then-prototype  format  order  condition  had  a  higher  MST  than  the  prototype-then-NOS 
format  order  condition),  and  it  would  explain  the  consistent  direction  of  the  differences 
between  the  formats  in  both  format  order  conditions  (prototype  was  faster  in  both  format 
order  conditions).  These  causes  would  also  explain  the  difference  in  the  magnitude  of  the 
between-format  difference  within  each  format  order  condition  (a  bigger  difference  for  the 
NOS-then-prototype  format  order  condition  than  for  the  prototype-then-NOS  format  order 
condition). 

The  second  hypothesis  stated  that  the  iconic  missed  approach  instructions  would  elicit  faster 
and  more  accurate  information  retrieval  than  the  text  instructions.  The  test  of  this  hypothesis 
was  provided  by  the  analyses  for  the  missed  approach  instruction  question.  Pilots  were  asked 
"In  the  missed  approach  procedures,  what  is  the  [first  or  second  heading  or  first  or  second 
altitude]?"  As  shown  in  Tables  11  and  12  and  Figure  14,  the  MST  required  by  pilots  to 
answer  the  missed  approach  question  was  significantly  faster  for  the  prototype  format  than  for 
the  NOS  format. 

The  missed  approach  question  pertained  to  the  "up  and  out"  portion  of  the  missed  approach 
instructions.  Both  charts  provided  the  full  instructions  in  text.  On  the  prototype,  this  text  was 
placed  in  the  briefing  strip,  while  on  the  NOS,  the  text  appeared  in  the  profile  view.  Only  the 
prototype  provided  the  up  and  out  portion  in  icons.  In  light  of  the  pilots'  comments  given  in 
Appendix  I,  the  faster  MST  for  the  prototype  format  can  be  attributed  to  the  icons.  Because  of 
the  icons,  plots  unanimously  preferred  the  prototype  chart  for  executing  the  missed  approach 
procedures.  These  results  provide  strong  support  for  the  second  hypothesis. 

The  third  hypothesis  predicted  that  overall,  the  prototype  chart  would  produce  faster  and  more 
accurate  information  transfer  than  the  NOS  chart.  This  hypothesis  was  tested  by  the  analysis 
for  answering  all  questions  shown  in  Table  3.  As  presented  in  Tables  7  and  8,  the  MST 
required  by  pilots  to  answer  questions  was  significantly  faster  with  prototype  charts  than  with 
NOS  charts.  Figure  12  shows  that  the  MST  for  the  prototype  format  was  approximately  nine 
seconds  faster  than  MST  for  the  NOS  format.  The  data  provide  strong  support  for  the  third 
hypothesis.  This  result  would  be  expected  since  12  of  the  16  questions  pertained  to  items  in 
the  prototype's  briefing  strip,  and  the  faster  MSTs  shown  for  that  format  have  been  discussed 
above. 

The  fourth  hypothesis  stated  that  pilots  would  prefer  the  prototype  format  to  the  NOS  format. 
Pilot  comments  were  used  to  test  this  hypothesis.  These  comments  are  provided  in  Appendix  I 
and  summarized  in  Table  13.  Pilots  were  asked  whether  they  preferred  one  of  the  chart 
formats,  or  had  no  preference  for  performing  the  following:  answering  the  quick  reference 
question,  reviewing  the  approach,  executing  the  approach,  and  executing  the  missed  approach. 


The  NOS  chart  was  never  preferred  by  any  pilot  for  performing  any  of  the  tasks.  The 
prototype  chart  was  unanimously  preferred  for  answering  the  quick  reference  question  and  for 
executing  the  missed  approach  procedures.  Eight  of  the  10  pilots  preferred  the  prototype 
format  for  studying  for  the  approach.  Six  pilots  preferred  the  prototype  format  for  executing 
the  approach.  Pilots  were  probably  searching  through  the  plan  and  profile  views  while 
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executing  the  approach,  and  these  sections  are  quite  similar  in  both  formats.  That  is  why  the 
very  strong  preference  for  the  prototype  chart  which  was  shown  in  the  other  approach  phases 
was  not  as  strong  for  this  phase.  Note  that  while  four  pilots  did  not  express  a  strong 
preference  for  either  format  when  executing  the  approach,  the  NOS  format  was  never  chosen. 

Pilots  were  also  asked  to  list  the  features  of  the  prototype  chart  that  they  did  not  use.  The 
purpose  of  this  question  was  to  determine  if  specific  features  of  the  prototype  were  not  useful 
and/or  should  be  redesigned.  Another  reason  for  this  question  was  to  determine  if  the  pilots 
took  advantage  of  the  prototype's  features,  or  whether  they  ignored  the  design  differences  and 
used  it  like  the  NOS  chart.  Both  the  MST  data  and  pilot  comments  indicate  that  pilots  took 
advantage  of  the  prototype’s  features. 

An  interesting  theme  in  the  comments  is  shown  in  Table  14.  Every  feature  mentioned  is 
located  in  the  briefing  strip.  The  items  in  the  briefing  strip  that  pilots  most  often  cited  as  not 
being  used  were  the  airport  elevation,  final  approach  fix  information,  and  final  approach 
course.  This  is  not  to  say  that  they  did  not  need  this  information,  only  that  those  particular 
pilots  did  not  refer  to  the  briefing  strip  for  those  items. 

Examination  of  the  comments  in  Appendix  I  and  the  NOS  format  shown  in  Appendix  D 
reveals  that  these  three  features  already  have  consistent  locations  on  the  NOS  chart.  The 
airport  elevation  is  always  boxed  in  the  upper  left  comer  of  the  airport  sketch.  Every  pilot 
who  cited  this  feature  said  that  he/she  looked  at  the  airport  sketch  for  that  information.  The 
final  approach  fix  information  and  final  approach  course  are  consistently  placed  in  the  center 
of  the  profile  view.  Every  pilot  who  cited  these  features  said  he/she  looked  at  the  profile  view 
for  these  items.  These  pilots  did  not  take  advantage  of  the  briefing  strip  formatting  because 
they  were  used  to  looking  someplace  else.  The  prototype  format  still  allowed  them  to  do  this, 
by  retaining  the  current  placement  of  those  items  in  the  airport  sketch  and  profile  view,  as 
well  as  in  the  briefing  strip. 

Finally,  pilots  were  asked  another  open-ended  question  -  if  they  had  any  other  comments. 
These  comments  are  provided  in  Appendix  I  and  summarized  in  Table  15.  All  of  the 
comments  pertained  to  the  prototype  chart.  Twenty  six  of  the  28  comments  were  positive. 

In  summary,  the  pilots'  comments  indicate  an  overwhelming  preference  for  the  prototype 
format.  Very  strong  support  was  found  for  the  hypothesis  that  pilots  would  prefer  the 
prototype  format  to  the  NOS  format.  In  most  cases  this  preference  for  the  prototype  format 
was  associated  with  improved  performance  on  the  objective  measures. 


4.2  SUMMARY  OF  MAJOR  FINDINGS 

•  Pilots  used  the  prototype's  briefing  strip  to  search  for  information  to  answer  questions 
pertaining  to  the  content  of  the  briefing  strip.  They  found  the  information  faster  this  way 
than  when  they  used  the  NOS  format  to  answer  those  questions. 

•  When  pilots  searched  for  information  on  the  charts  in  order  to  answer  all  the  questions, 
they  found  information  faster  on  the  prototype  chart  than  on  the  NOS  chart. 
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•  When  pilots  searched  for  information  on  the  charts  in  order  to  answer  all  the  questions,  no 
difference  was  found  between  the  accuracy  of  the  answers  given  when  pilots  used  the 
prototype  and  when  they  used  the  NOS  format. 

•  Pilots  found  information  regarding  the  "up  and  out"  portion  of  the  missed  approach 
instructions  faster  when  they  used  the  iconic  missed  approach  instructions  on  the  prototype 
chart  than  when  they  used  text  instructions  on  the  NOS  chart. 

•  Most  pilots  preferred  the  prototype  format  for  reviewing  the  approach. 

•  AH  pilots  preferred  the  prototype  format  for  executing  the  missed  approach.  Their 
preference  was  due  to  the  use  of  icons  to  depict  the  up  and  out  portion  of  the  missed 
approach  instructions. 

•  None  of  the  pilots  preferred  the  NOS  format  for  executing  any  approach  phase. 


4.3  CONCLUSION 

Pilots  were  able  to  find  information  faster  on  the  Voipe  prototype  chart  than  on  the  NOS 
chart,  and  they  indicated  a  clear  preference  for  the  prototype  format  over  the  NOS  format. 
These  findings  are  consistent  with  the  outcome  of  past  laboratory  experimentation  (Multer, 
Warner,  DiSario,  and  Huntley,  1990;  Osborne  and  Huntley,  1992),  subject  matter  expert 
interviews  (Osborne  and  Huntley,  1993),  a  review  of  the  relevant  literature  (Mangold, 
Eldredge,  and  Lauber,  1992),  and  a  field  evaluation  (Blomberg,  Bishop,  and  Hamilton,  in 
press). 

Based  on  all  of  the  above  findings,  the  National  Ocean  Service  is  strongly  urged  to  adopt  the 
prototype  format  for  its  lAP  charts. 
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APPENDIX  A 


INFORMED  CONSENT  FORM 


Informed  Consent  Form 


You  have  been  asked  to  participate  in  the  Prototype  Instrument  Approach  Procedure  Chart 
Study  conducted  by  the  Operator  Performance  Division  of  the  Volpe  National  Transportation 
Systems  Center  (VNTSC).  The  study's  purpose  is  to  assess  the  speed  and  accuracy  with  which 
pilots  are  able  to  read  information  from  different  kinds  of  approach  chart  designs.  If  you  agree 
to  participate,  you  will  be  asked  to  participate  for  two  days.  On  one  day  you  will  fly  a 
Beechcraft  Baron  BE55  aircraft,  and  on  the  other  day  you  will  fly  a  Frasca  flight  simulator. 

During  both  flying  sessions,  you  will  fly  a  series  of  GPS  approaches  with  different  kinds  of 
approach  charts.  After  reviewing  the  information  in  your  usual  method,  you  will  be  asked  a 
series  of  questions  regarding  information  found  on  the  approach  chart.  In  order  to  answer  the 
question,  you  will  be  able  to  answer  from  memory  or  push  a  switch  to  open  a  shutter  and 
read  the  chart.  As  you  fly  the  simulator  and  the  Baron,  we  will  score  your  crosstrack  error 
and  deviations  from  desired  altitude  and  airspeed.  You  will  also  be  asked  to  complete  short 
questionnaires  regarding  your  flight  experience  and  instrument  approach  procedure  chart  use. 
After  the  flight  session  has  been  completed,  you  will  be  asked  for  your  opinions  on  the 
different  chart  designs. 

You  will  have  the  choice  between  flying  the  approaches  under  a  hood  or  with  foggles.  All 
approaches  will  be  flown  at  Gardner  Municipal  Airport  (GDM).  The  charts  you  will  use  in  the 
airplane  (and  in  the  simulator)  will  depict  data  that  is  not  approved  for  IFR  approaches, 
however,  there  will  be  a  safety  pilot  in  the  Baron.  If  the  safety  pilot  becomes  incapacitated, 
remove  the  instrument  hood  or  foggles  and  assume  pilot-in-command  status  for  flying  the 
aircraft.  A  full  set  of  accurate  instrument  approach  charts  for  GDM,  Hanscom  (BED)  and  the 
surrounding  airports  will  be  available  to  you  at  all  times. 

The  experiment  will  take  approximately  8  hours  each  day  to  complete.  Please  do  not  hesitate 
to  ask  questions  about  the  study  at  any  time.  Your  data  will  be  kept  strictly  confidential  and 
your  name  will  not  be  associated  with  your  data.  Your  participation  in  this  study  is  strictly 
voluntary.  If  you  agree  to  participate,  and  you  are  a  Volpe  Center  employee,  you  will  be 
given  a  VNTSC  account  number  to  which  you  may  charge  your  time.  If  you  are  not  a  Volpe 
Center  employee,  your  expenses  Avill  be  reimbursed  according  to  the  terms  stated  on  the 
reimbursement  form.  You  are  free  to  withdraw  at  any  time  without  penalty,  and  your 
participation  is  sincerely  appreciated. 


Signature  and  Age  of  Participant  Signature  of  researcher 


Name  (please  print)  Date 


Address  and  Daytime  Phone  Number 
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APPENDIX  B 


FLIGHT  EXPERIENCE  AND  CHART  PREFERENCE  QUESTIONNAIRE 


Pilot  Questionnaire 


Thank  you  for  participating  in  this  research  -  your  contribution  is  vital.  Unlike  the  vast  majority 
of  aviation  human  factors  research,  much  of  the  work  conducted  here  at  the  Volpe  Center  Cockpit 
Human  Factors  Lab  is  specifically  focused  on  the  general  aviation  pilot.  This  research  is  paid  for 
by  the  FAA  and  the  results  will  provide  a  basis  for  making  decisions  that  will  increase  air  safety 
for  all  pilots  and  for  general  aviation  pilots  in  particular.  It  could  not  be  accomplished  without 
pilots  like  you  and  we  sincerely  appreciate  your  participation. 

Your  name  will  not  be  associated  with  your  data.  However,  we  do  need  to  ask  you  for  some 
descriptive  information  since  it  may  help  us  later  on  when  we  are  studying  trends  in  the  data. 


1.  Age:  _  Gender:  Male 

2.  Approximately,  what  is  your:  total  flight  time 

total  instrument  time 


Female 

_____  hours 
hours 


3.  Please  indicate  your  areas  of  aviation  experience,  and  for  each  area,  please  circle  the  approach 
category(ies)  of  aircraft  most  often  flown: 

Military  A  B  C  D  E  Part  121  BCD  Part  135  ABC 

General  Aviation  ABC  Corporate  A  B  C  D 


4.  Please  estimate  the  percentage  of  time  you  use  the  following  instrument  approach  procedure  charts: 

Jeppesen  _ % 

NOAA  _ % 

Other  (please  identify) _ % 


5.  Please  rank  your  preference  for  these  manufacturers'  instrument  approach  procedure  charts  (1  —  most 
preferred,  3  =  least  preferred  -  if  you  do  not  use  one  of  these  types  of  charts,  write  "NA"): 

Jeppesen  _ 

NOAA  _ 

Other  _ 
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APPENDIX  C 

DESCRIPTIONS  OF  THE  FLIGHT  PROCEDURES  FOLLOWED  IN  THE 
SIMULATOR  AND  AIRPLANE 


Pilot  Brief  -  Frasca  Flight  Simulator  Procedures 

Your  task  will  be  to  fly  instrument  approaches  using  either  NOS  or  prototype  lAP  charts. 
After  having  an  opportunity  to  review  the  approach  you  will  fly  the  procedure,  and  the 
experimenter  will  ask  you  a  series  of  questions  regarding  information  on  the  chart.  All 
approaches  will  be  flown  at  Gardner  Municipal  Airport  (GDM). 

During  the  approaches,  course  guidance  will  be  provided  by  a  simulated  GPS  receiver  display 
presented  on  a  CRT  in  the  center  of  the  instrument  panel.  The  simulated  receiver  will  drive 
the  CDI  on  the  HSI  in  front  of  you.  Needle  sensitivity  will  be  set  to  3/10  nm  full  scale.  The 
receiver's  display  will  show  the  waypoint  you  are  flying  to,  the  distance  to  that  waypoint,  and 
the  bearing  to  that  waypoint. 

To  begin  each  approach,  the  experimenter  will  position  you  approximately  12  nm  from  the 
initial  approach  fix.  You  will  be  given  the  chart  to  review  while  you  are  flying  to  the  initial 
approach  fix. 

As  you  approach  the  missed  approach  point  (MAP)  the  experimenter  may  instruct  you  to  land 
If  you  are  instructed  to  land,  remove  your  foggles/hood,  slow  to  90  knots,  and  descend  to  the 
field  elevation.  If  he  does  not  instruct  you  to  land,  fly  the  missed  approach  procedures  as 
depicted  on  the  approach  chart.  Reconfigure  the  aircraft  as  usual  for  the  climb,  with  a  target 
airspeed  of  120  knots. 

Fly  to  the  same  standards  you  trained  to  for  your  instrument  practical  test.  Fly  each  approach 
at  120  knots  ±10  knots,  and  maintain  the  altitudes  ±100  feet.  Do  not  descend  below  the 
MDAs  shown  for  Category  B  aircraft. 

Some  useful  approximations  for  maintaining  120  knots  with  the  Frasca: 

level  flight  -  gear  up,  flaps  up;  RPM  2400  MP  16  -  17" 

level  flight  -  gear  down,  flaps  50%:  RPM  2700  MP  18" 

1000  fpm  descent  -  gear  down,  flaps  50%:  RPM  2700  MP  15" 


Pilot  Brief  -  Baron  Flight  Procedures 

Today's  flight  procedures  are  basically  the  same  as  those  used  during  your  session  in  the 
Frasca.  Your  task  will  be  to  fly  instrument  approaches  using  either  NOS  or  prototype  lAP 
charts.  After  having  an  opportunity  to  review  the  approach  you  will  fly  the  procedure,  and  the 
experimenter  will  ask  you  a  series  of  questions  regarding  information  on  the  chart.  All 
approaches  will  be  flown  at  Gardner  Municipal  Airport  (GDM). 

During  the  approaches,  course  guidance  will  be  provided  by  a  Garmin  155  TSO  GPS  receiver. 
The  safety  pilot  will  be  responsible  for  operating  the  GPS  receiver.  The  receiver  has  its  own 
CDI  and  will  also  drive  the  CDI  on  the  HSI  in  front  of  you.  Needle  sensitivity  for  both  the 
receiver's  CDI  and  the  CDI  on  your  HSI  will  be  set  to  3/10  nm  full  scale.  The  receiver's 
display  will  show  the  waypoint  you  are  flying  to,  the  distance  to  that  waypoint,  and  the 
bearing  to  that  waypoint. 

To  begin  each  approach,  the  safety  pilot  will  give  you  vectors  that  will  allow  you  to  intercept 
the  approach  course  and  cross  the  initial  approach  fix  on  course.  While  you  are  flying  these 
vectors,  you  will  be  given  the  chart,  and  you  may  then  begin  your  review  of  the  approach. 

As  you  approach  the  missed  approach  point  (MAP)  the  experimenter  may  instruct  you  to  land. 
If  you  are  instructed  to  land,  remove  your  foggles/hood,  slow  to  90  knots,  and  continue  the 
approach  visually  to  a  full  stop  landing.  If  he  does  not  instruct  you  to  land,  fly  the  missed 
approach  procedures  as  depicted  on  the  approach  chart.  Reconfigure  the  aircraft  as  usual  for 
the  climb,  with  a  target  airspeed  of  120  knots. 

Fly  to  the  same  standards  you  trained  to  for  your  instrument  practical  test.  Fly  each  approach 
at  120  knots  ±10  knots,  and  maintain  the  altitudes  ±100  feet.  Do  not  descend  below  the 
MDAs  shown  for  Category  B  aircraft.  If  you  exceed  these  standards,  the  safety  pilot  will 
bring  it  to  your  attention. 

If  the  safety  pilot  tells  you  that  he  has  control  of  the  aircraft,  you  should  give  him  the 
controls  and  tell  him  he  has  control. 

If  the  safety  pilot  becomes  incapacitated,  remove  the  foggles/hood  and  assume  pilot-in- 
command  status.  A  full  set  of  current  lAP  charts  will  be  available  to  you  at  all  times. 


Some  useful  approximations  for  maintaining  120  knots  with  the  Baron: 


level  flight  -  gear  up,  flaps  up; 

RPM  2300  -  2500 

MP  15" 

level  flight  -  gear  down,  flaps  15°; 

RPM  2300  -  2500 

MP  17" 

1000  fpm  descent  -  gear  down,  flaps  15°: 

RPM  2500 

MP  15" 

C-2 


APPENDIX  D 

CHARTS  AND  APPROACH  LIGHTING  SYSTEM  SKETCHES  USED  FOR  TRAINING 


VOLPE  CENTER  CHART-NOT  FOR  NAVIGATION 

Call  (617)  494-2362  or  2339  with  questions  or  comments 


Arndt  15  920098 


VOR  RWY  25R  (GPS) 


•2126 

1729 


1662 

.  1756  A 

- lAF - 

SANTA  MONICA 

110  8  SMO  _ 

Chan  45 

863  ^ 

^  5000 

068"  (19.1) 


AL-237  (FAA) 


2035  ^ 


4000*  to  Fiton  Int 
161"  (7.8)  and 
248"  (3.2) 


LOS  ANGELES  INTL(LAX) 

LOS  ANGELES.  CALIFORNIA 


ATiSARR  133.8 
DEP  135.65 
LOS  ANGELES  APP  CON 
124  5  381.6 
LOS  ANGELES  TOWER 
CAP)  N  133.9  239.3 

is  120.95  379.1 
GND  CON 
N  121.65  327.0 
S  121.75  327.0 
CLNC  DEL 
121.4  327.0 


SMOGY  INTt- 

SMOn?Dt 


4000*  to  Fiton  Int 
161"  (4.7)  and 
248"  (3.2) 


LOS  ANGELES 
me  LAX 
Chan  83 


A 

\  2000 

.MM 

-W248^- 

OM 

V 

(7.5) 

- R- 

/ 

A 

1310 

/  iJJfSb 

4000*  to  Fiton  Int 
256"  (9.7)  and 
325"  (2) 


*3300  whon  authorized 
by  ATC 


- - H-251- 

^  585 


- iAF - ^ 

SEAL  BEACH 
11S.7SLI  ^ 
Chan  104 


MISSED  APPROACH 

Clirr*)  to  600  then  climbing  left 

FITON 

turn  to  2000  via  heading  220" 

INT 

and  LAX  R-192  then  clirrtb  to 

3000  to  INISH  InVLAX  12  DME. 

FREBY 

INT  ^  OftO 
- 

4000* 

NOELE 

INT 

1 

1 

ANGEL  1  ^ 

1  2000 

1 

\  T 

1 

1 

''-_UH620 

1 

1 

1  1 

1 

CATEGORY 


S-25R** 


620/24  519  (500-1/2) 


620/24  518  (500-1/2) 


620/50 

519  (500-1) 


620/50 

518  (500-1) 


620/60 
519  (500-11/4) 


620/60 
518  (500-11/4) 


**Cat.  D  visibility  increased  to  RVfl  6500  for  irx)perative  ALSF-2. 
ADF  or  radar  required. 


VOR  RWY  25R  (gps) 


33«57*N-118'’24*W 


IilWEgiEliBaiBHIEEMl 


LOS  ANGELES,  CALIFORNIA 

LOS  ANGELES  ItJTL  (LAX) 


VOLPE  CENTER  CHART-NOT  FOR  NAVIGATION 

Call  (617)  494-2362  or  2339  with  questions  or  comments 


APP  CRS 

FAF 

TDZE 

APT  ELEV 

FREBY 

24B° 

2000*  (1874’) 

102* 

126* 

*3300  wh»n  authonzsd  by  ATC 

•*Cat.  D  VHitxIrty  mcrsaawd  to  RVR  6500  for  iriop«rattv0  ALSF*2. 
ADF  or  radar  required 


ATIS  LOS  ANGELES  APR  LOS  ANGELES  TOWER.  GROUND  CLNC  DEL 

Arr  Dep  North  South  North  South 

133.8  135.65  124.5  381.6  133.9  239.3  133.9  239.3  121.65  327.0  121.75  327.0  121.4  327.0 


1862 

,  1756  A 

- lAF - 

SANTA  MONICA 

110.8  SMO  — _ 

Chan  45 _ 

1  5000 

068^(19.1) 


(lAF) 

4000*  to  Fiton  Int  ^>-MOO 

16r  {7.8}  and  _ ^ 

248"  (3.2) 


FREBY  INT\  fiton  INT  k 


-  LOS  ANGELES  INTL  (KLAX) 

LEV  LOS  ANGELES.  CALIFORNIA 

5-  VOR  RWY  25R  (gps) 


ALSF-2  missed  APPROACH:  Clinib  to  600  then  dirrbing  left 
z  E  5  turn  to  2000  via  heading  220"  and  LAX  R-1 92  then 
4^  clirrtj  to  3000  to  INISH  Int/LAX  1 2  DME. 


^VOR 

^  _ _ _ 

12900  I 


SMOGY  INTI 
SMO|i9  t)  1 


'  4000*  to  Fiton  Int 
161"  (4.7)  and 
248"  (3.2) 


^  ^  4000*  to  Fiton  Int 

Vt  256"  (9.7)  and 
325"  (2) 


- lAF - ^ 

SEAL  BEACH 
115.7  SLl 
Chan  104 


ELEV  126  Rwy  25 R  Idg  11134’ 

- '  Rwy  6R  Idg  9964* 


A 

I  8925X1^1 


248"  5.5  NM 
from  FAF 


©242^^’^'^  289  305°^ 
101 


VOR  RWY  25R  (GPS) 


620/50  620/60  TDZ/CL  Rwys  6R,  24R  and  25L 

518  (500-1)  518  (500-1’/,)  HIRL  all  Rw^ 


92233  33‘’57  N-1 1  e‘‘24Vi  LOS  ANGELES  INTL  (KLAX) 

AL-237  (FAA)  LOS  ANGELES.  CALIFORNIA 


D-2 


TYPE  1 


TYPE  2 


TYPE  3 


APPENDIX  E 

TRAINING  GUIDES  FOR  BOTH  THE  NOS  AND  PROTOTYPE  FORMAT  CHARTS 


Briefing  Guide  for  the  NOS  Format  Instrument  Approach  Procedure  Chart 
[Use  the  NOS  LAX  GPS  Overlay  VOR  Rwy  25R  as  the  example.] 

As  an  IFR  pilot,  you  are  already  very  well  practiced  in  interpreting  the  content  of  lAP  charts. 
Even  though  you  are  currently  familiar  with  the  NOS  design,  it's  important  to  review  the 
location  of  key  information. 

[During  the  review,  actually  point  to  each  of  the  information  elements  listed  below.  Describe 
the  location  and  content  of  each  element.] 


Plan  View 

•  communication  frequencies 

•  approach  course 

•  location  of  information  (type,  identifier,  and  frequency)  for  the  navaid  that  serves  the 
approach 

•  MSA  circle 

•  identification  of  the  reference  navaid  on  the  perimeter  and  in  the  center  of  the  circle 

•  segments  defined  by  bearing  from  the  navaid 

•  missed  approach  holding  fix 


Profile  View 

•  missed  approach  instructions 

•  approach  course 

•  MSL  altitude  at  the  FAF 


Minimums 

•  MDAs 


Notes 

•  notes  are  given  in  both  this  section  and  in  the  plan  view 


Airport  Sketch 

•  touchdown  zone  elevation 

•  airport  elevation 

•  approach  lighting  system  sketch 


E-1 


Brieflng  Guide  for  the  Prototype  Format  Instrument  Approach  Procedure  Chart 

[Use  the  prototype  LAX  GPS  Overlay  VOR  Rwy  25R  as  the  example.] 

Background  The  purpose  of  the  prototype  is  to  improve  the  speed  and  accuracy  with  which 
pilots  locate  and  comprehend  information  on  instrument  approach  procedure  (lAP)  charts.  It  is 
an  attempt  to  change  the  formatting  of  the  information  to  conform  more  closely  to  the  way 
pilots  actually  use  the  information.  This  format  was  developed  from  a  combination  of 
laboratory  experimentation  conducted  by  the  Cockpit  Human  Factors  Program  here  at  the 
Volpe  Center,  a  field  evaluation  conducted  at  the  training  centers  of  four  airlines,  and  several 
in-depth  subject  matter  expert  reviews. 

Features  of  the  Prototype 

1.  Briefing  Strip  -  These  three  rows  of  information  are  used  for  preparing  for  the  approach. 
The  location  of  each  information  element  is  standardized  and  given  in  the  order  in  which  it 
will  be  used.  In  the  first  and  third  lines  of  the  strip,  the  information  to  be  briefed  is  given  in 
bold  type,  while  the  name  of  the  information  is  shown  in  regular  type.  The  pilot  no  longer  has 
to  search  through  the  entire  chart  to  assemble  this  data. 

1.1.  The  information  required  for  quick  reference  is  in  the  top  row.  The  first  box 
contains  the  type,  identifier,  and  frequency  of  the  navaid  that  serves  this  approach.  [Point  out 
the  contents  of  the  other  boxes.]  If  the  charted  procedure  were  a  non-precision  approach  to 
either  of  two  runways,  both  TDZEs  would  be  shown,  and  both  will  be  bolded.  To  the  right  of 
this  row,  the  four-letter  ICAO  airport  identifier  is  shown  in  small  italics. 

1.2.  The  second  row  contains  all  equipment  and  procedural  notes  (navigation  notes 
remain  in  the  profile  section),  an  approach  lighting  system  sketch,  and  missed  approach 
instructions.  If  any  visual  glideslope  indicators  are  present,  they  are  also  depicted  in  the 
approach  lighting  system  sketch.  If  a  parallel  runway  has  the  same  lighting  system,  an 
italicized  note  is  given  in  the  notes  box. 

1.3.  The  ATIS  and  communication  frequencies  are  listed  in  order  of  their  use  and  the 
numbers  are  always  shown  under  the  labels.  In  order  to  save  space,  they  are  not  as  large  as 
the  numbers  listed  above.  They  are  made  available  here  for  verification,  early  radio  set-up,  or 
in  case  of  blockage  of  an  active  communication  channel.  Since  the  plan  view  is  oriented 
North-up,  the  West  and  East  frequencies  are  placed  on  the  left  and  right,  respectively. 


2.  MSA  Circle  -  The  MSA  circle  will  float  within  the  plan  view  to  occupy  unused  space. 

2.1.  The  text  identification  for  the  reference  navigational  aid  (navaid)  is  given  in  the 
center  of  the  circle. 

2.2.  The  sectors  of  the  circle  are  defined  by  radials  rather  than  bearings,  since  pilots 
usually  think  in  terms  of  radials  rather  than  bearings  to  the  station. 


E-2 


I 


3,  Plan  View  - 


3.1  The  approach  course  number  has  been  boxed  to  enhance  detectability  and 
readability. 

3.2  The  name  of  the  missed  approach  holding  fix  has  been  bolded. 


4.  Profile  View  - 

4.1.  In  the  profile  view,  the  "up  and  out"  portion  of  the  missed  approach  instructions 
is  depicted  in  icons  rather  than  text.  These  icons  tell  the  pilot  all  that  is  required  to  get  the 
plane  up  and  out  -  and  this  critical  information  is  more  easily  located  than  if  it  were 
embedded  in  text.  The  first  altitude  and  navaid  frequency  are  emphasized  by  bolding.  If  there 
was  a  heading  in  the  first  box,  it  too  would  be  bolded. 

4.2  The  approach  course  number  has  been  boxed  here  as  well. 


5.  Minimums  - 

5.1  Bold  type  has  been  used  to  make  the  MDAs  easier  to  find  and  read. 


APPENDIX  F 

TRAINING  MATERIAL  FOR  ICONIC  MISSED  APPROACH  INSTRUCTIONS 


The  Use  of  Icons  for  Missed  Approach  Instructions 


Climb  to  4000 


4000 

t 


Climb  to  6000  then  climbing  left  turn  to  8000 


6000 

8000 

f 

Note  that  the  first  altitude  (6000)  is  in  bold  print. 


Climb  to  8000  heading  110° 


8000 

110° 


t 


Climb  to  9000  heading  070°  then  climbing  left 
turn  to  10000  heading  040° 


9000 

10000 

070° 

040° 

t 

Note  that  both  the  first  altitude  (9000)  and  the 
first  heading  (07CP)  are  in  bold  print. 

F-1 


Climb  to  3000  heading  240°  then  climbing 
right  turn  to  5500  heading  290° 


3000 

5500 

240° 

290° 

f 

r 

Climb  to  8000  to  intercept  CEL  R-330 
(112.7  is  the  frequency  for  CEL) 

Note  that  the  radial  (R-330)  and 
frequency  (1 12. 7)  are  in  bold  print. 


8000 

CEL 

t 

R-330 

Climb  to  7000  to  intercept  PAR  R-300 


7000 

j  PAR 

t 

•  R-300 

[  115.6 

Climb  to  8000  then  climbing  left  turn  to  9000 
to  intercept  JEN  R-150 


8000 

9000 

JEN 

t 

fl-150 

lUIL. 

Climb  to  3000  then  climbing  right  turn  to  3800 
to  intercept  PIN  R-070 


3000 

3800 

1  PIN 

f 

,  R-070 

r 

1 

1  118.1 

F-2 


Climb  to  4500  heading  1 00°  then  climbing  left 
turn  to  6000  via  heading  042°  and  GAM  R-260 


4500 

6000 

‘  GAM 

100° 

042° 

J  n-260 

f 

,  110.9 

Climb  to  2000  heading  128°  then  climbing  right 
turn  to  3300  heading  155°  to  intercept  BIR  R-330 


2000 

3300 

'  SIR 

128° 

155° 

'  B^30 

t 

1 

,  112.6 

Climb  to  6000  heading  040°  then  climbing  right 
turn  to  7800  heading  080°  to  intercept  MAZ  R-260 


6000 

7800 

‘  MAZ 

040° 

080° 

!  R-260 

f 

r 

1 

,  113.9 

Climb  to  4000  heading  135°  then  climbing  left 
turn  to  6300  heading  1 09°  to  intercept  KAN  R-305 


4000 

6300 

KAN 

135° 

109° 

R-305 

f 

!  110.3 

Climb  to  5000  heading  1 08°  then  climbing  right 
turn  to  7200  heading  1 15°  to  intercept  KER  R-310 


5000 

7200 

KER 

108° 

115° 

R-310 

t 

r 

,  117.2 

F-3/F-4 


APPENDIX  G 

SELF-TEST  FOR  ICONIC  MISSED  APPROACH  INSTRUCTIONS 


Please  read  this  before  continuing:  Use  this  page  to  determine  whether  or  not  the  icons  are 
clear  to  you.  Interpret  the  icons  before  moving  the  card  to  check  the  text. 


Climb  to  8400  heading  355°  then  climbing  left 
turn  to  10,000  heading  230°  to  intercept  REG  R-135 


8400 

10000 

'  REG 

355“ 

230“ 

[  R-135 

t 

1  116.S 

Climb  to  3200  heading  165°  then  climbing  right 
turn  to  5800  heading  220°  to  intercept  RAJ  R-077 


3200 

5800 

1  RAJ 

165“ 

220“ 

[  R^77 

t 

r 

1  114.2 

Climb  to  9200  heading  140°  then  climbing  left 
turn  to  1 1 ,000  heading  125°  to  intercept  DUG  R-250 


9200 

11000  I 

'  DUG 

140“ 

125°  i 

j  R-250 

t 

i 

I  107.8 

Climb  to  2100  heading  278°  then  climbing  left 
turn  to  5000  heading  253°  to  intercept  HAL  R-323 


2100 

5000 

'  HAL 

278“ 

253“ 

J  R-323 

t 

I  113.5 

Climb  to  3700  heading  097°  then  climbing  right 
turn  to  4800  heading  190°  to  intercept  DIN  R-014 


3700 

4800 

DIN 

097“ 

190° 

R-014 

t 

1 

G-l/G-2 


APPENDIX  H 

TRAINING  GUIDE  FOR  LIGHT  SHUTTER  OPERATION 


Briefing  Guide  for  the  Liquid  Crystal  Light  Shutter 


Throughout  the  study,  you  will  be  looking  for  information  on  the  charts.  We  need  to  keep 
track  of  how  often  and  for  how  long  you  look  at  the  charts,  and  this  light  shutter  allows  us  to 
do  that.  It  is  controlled  by  the  push-to-talk  type  switch  which  has  been  Velcroed  to  the  left 
yoke  handle. 

A  chart  will  be  placed  under  the  shutter.  When  you  want  to  open  the  shutter  and  look  at  the 
chart,  press  the  switch  [demonstrate]. 

[Allow  the  pilot  to  open  and  close  the  shutter  several  times.  Ask  the  pilot  if  the  switch  is  in  a 
place  that's  comfortable  and  easy  to  reach.  Make  your  own  assessment  of  the  placement  of  the 
switch.  Does  the  pilot  appear  to  be  comfortable?  Does  the  switch  appear  to  be  easy  to 
reach?] 


During  the  experiment,  you  may  look  at  the  chart  whenever  you  need  to.  However,  we  want 
to  emphasize  that  you  should  minimize  the  number  of  looks,  and  the  length 
is  a  very  important  point.  You  may  look  at  the  chart  whenever  you  need  to, 
look  at  the  chart  try  to  minimize  the  number  of  times  you  look  at  the  chart, 
the  length  of  each  look  to  a  minimum.  By  minimizing  both  the  number  and 
you  will  be  helping  us  to  evaluate  the  design  of  the  charts. 

Do  you  have  any  questions? 


of  each  look,  this 
but  when  you 
and  try  to  keep 
length  of  looks. 


H-l/H-2 
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PILOT  COMMENTS 


Pilot  Comments 


Comments  from  each  of  the  ten  pilots  are  organized  by  topic.  The  experimenters  questions 
are  shown  in  bold  italics.  The  authors  edited  the  grammar  in  some  of  the  comments  to  make 
them  easier  to  read.  In  some  cases,  words  were  inserted  to  clarify  the  comments.  These  edits 
and  insertions  are  shown  in  brackets. 


'Which  chart  did  you  prefer  for  answering  the  first  [quick  reference!  question  (frequency  of 
the  VOR,  approach  course,  altitude  at  the  final  approach  fix)  -  the  NOS,  the  prototype,  or  no 
preference?" 

New  [prototype]  chart.  All  information  [was]  in  order  when  asked  the  question.  Other  [NOS] 
chart  -  had  to  look  around. 

Prototype.  All  the  information.  The  most  important  information  has  a  consistent  location  on 
the  map.  Drawn  out  the  information. 

Prototype.  Because  everything  is  in  line.  Vital  information  is  in  the  top  line.  That's  the  highest 
priority  -  in  the  top  line.  You  really  need  that  information. 

Prototype.  Easier  because  it  identifies  the  facility  for  the  approach  and  the  third  [altitude  at 
the  FAF]  box.  Approach  course  is  easy  to  find  on  either  chart,  so  that  wasn't  a  big  addition. 

Definitely  the  prototype  was  better,  because  it  was  right  there  in  a  line  for  you.  Didn't  have  to 
look  in  more  than  one  spot. 

Prototype.  Information  was  easily  found  and  easy  to  read. 

Prototype  was  better  because  it  was  an  easier  sequence  to  get  the  information  off  of  the  chart. 

Prototype,  because  the  essential  information  is  on  the  top  of  the  chart,  you  don't  have  to  scan 
ail  over  the  chart  to  find  out  what  you  need. 

Prototype  -  definitely.  Because  the  information  is  much  more  easily  accessible  on  the  top  line 
Definitely  the  prototype,  because  the  information  was  more  easily  or  quickly  accessible. 


'Which  chart  did  you  prefer  for  reviewing  the  cpproach  -  the  NOS,  the  prototype,  or  no 
preference?" 

Neither  [no  preference].  I  did  like  how  the  approach  lighting  box  is  set  off.  That  is  helpful.  I 
would  never  be  able  to  find  it  in  the  NOS. 

Prototype.  Any  airport,  you  need  to  know  what  lights  to  look  for.  The  lighting  sketch  at  the 
top  helps.  Briefed  the  icons  easier. 


M 


Prototype.  Because  they  are  bunched  in  accordance.  You  don't  have  to  look  all  over  the  chart. 
Frequencies  are  straight  across.  Very  good. 

Prototype.  Notes  are  easier  to  find  and  I  always  read  the  notes  to  brief.  I  really  use  the  profile 
view  to  brief 

No  preference,  except  for  briefing  the  missed  approach.  It  was  much  easier  to  brief  the 
prototype. 

Prototype.  Because  the  information  I  need  you  can  find  more  easily  on  the  prototype  than  the 
NOS.  I  mainly  look  at  the  profile  view  [for  the]  inbound  course.  The  missed  approach 
instructions  were  easier  to  brief  because  of  the  icons.  The  [communication]  frequencies  were 
much  easier  to  find  across  the  top.  Notes  were  easier  to  find  at  the  top. 

NOS  because  I'm  used  to  it. 

Prototype,  because  the  essential  information  you  need  for  briefing  is  on  the  top  and  in  the 
missed  approach  instructions,  the  icons  are  much  more  clear. 

Prototype,  because  the  layout  was  more  clear  -  top  three  lines  and  briefing  the  missed 
approach  instructions.  I  like  the  notes  all  in  one  spot  instead  of  scattered. 

I  still  prefer  the  prototype,  because  I  felt  as  though  the  important  information  was  more  easily 
accessible  and  stood  out  more.  [It  was]  less  congested.  [When  I  was  reviewing  the  prototype] 

I  found  I  would  use  the  information  at  the  top  and  the  profile.  In  the  profile  it  was  much 
easier  to  brief  with  the  icons,  and  the  missed  approach  [depiction]  is  visual  [pictorial]  so  I 
don't  have  to  take  the  time  to  read. 


'Which  chart  did  you  prefer  for  executing  the  (pproach  -  the  NOS,  the  prototype,  or  no 
preference?" 

Prototype.  Final  approach  course  is  boxed  on  plan  view  &  clearer  which  navaid  the  approach 
is  based  [upon].  Clear  because  it  [final  approach  course]  is  in  the  quick  reference  line  first. 

No  preference. 

Prototype.  Boxes  &  bold  lettering  -  liked  that.  Bold  lettering  makes  information  stand  out. 
About  the  same  [no  preference]. 

No  preference.  Usually  use  only  the  profile  [view].  The  only  difference  was  box[ed  final 
approach  course]  &  box[ing  the  final  approach  course]  didn't  make  much  difference.  Can't  tell 
really  if  bolding  [the  MDA  and/or  missed  approach  holding  fix]  helped. 

No  preference. 
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Prototype.  I  like  how  the  approach  course  is  boxed  in.  Boxing  helps  in  the  profile  and  plan 
views.  Also  liked  the  bolding  [of  the  MDAs]  in  the  minimums. 

Prototype.  During  a  high  workload  situation,  the  information  is  much  more  clear.  I  like  the 
frequencies  where  they  are  [on  the  prototype],  the  only  thing  is  I  think  the  frequencies  should 
go  vertically,  but  in  boxes.  I  like  how  they  are  on  the  Jeppesen  charts. 

Prototype.  I  use  the  profile  view  to  execute  [the  approach].  Boxing  makes  the  approach  course 
stand  out  more.  Otherwise,  not  a  lot  of  difference. 

The  prototype  for  the  same  reasons  as  I  mentioned  before  -  [the  important  information  was 
more  easily  accessible  and  stood  out  more.  [It  was]  less  congested.  [When  I  was  reviewing 
the  prototype]  I  found  I  would  use  the  information  at  the  top  and  the  profile.] 


'Which  chart  did  you  prefer  for  executing  the  missed  approach  procedures  -  the  NOS,  the 
prototype,  or  no  preference?" 

Prototype.  Icons  were  really  really  good.  Always  used  the  icons.  Only  thing  that  wasn  t 
iconized  was  the  missed  approach  holding  point.  I  had  to  go  to  the  text  to  find  out  how  far 
out  you  fly  the  radial.  Have  to  look  somewhere  else  to  complete  the  missed  approach.  I 
would  like  to  see  another  icon  with  the  intersection  (INISH  LAX  12)  in  the  icon  row.  Then  I 
would  never  have  to  look  up  in  the  text  row  at  all. 

Prototype.  Big  difference.  Icons  are  much  more  useful.  Can  get  a  3D  [3  dimensional]  view  of 
what  you  need  to  do  in  the  blink  of  the  eye. 

Prototype.  The  best  contribution,  best  improvement  to  [the]  entire  chart  is  the  icons.  [They 
are]  very  clear. 

Prototype.  Really  like  the  icons.  Easier  to  quickly  get  information,  but  it  is  missing  two 
things.  [1.]  Needs  to  tell  you  which  way  to  turn  when  you  get  to  the  intercept.  Needs  to  tell 
you  what  to  do.  One  more  box  would  do  it.  [2.]  Would  like  the  name  of  the  missed  approach 
fix  [placed]  in  the  profile  view  at  the  tip  of  the  [missed  approach]  arrow. 

Prototype.  Without  a  doubt  the  icons  are  the  way  to  go. 

Definitely  the  prototype.  Altitudes  and  headings  much  easier  to  find  and  read.  Liked  the 
bolding.  Bolding  the  missed  approach  fix  bold  didn't  help  enough.  It  didn't  stand  out  enough. 

Prototype.  Because  I  think  the  icons  help  you  pull  the  information  quickly,  rather  than  having 
to  read  it  off  the  plate. 

I  prefer  the  prototype.  The  icons  make  it  much  easier  to  obtain  the  information. 
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I  really  like  the  icons,  but  I  [would]  also  like  the  text  in  with  the  icons.  I  felt  like  I  had  to 
look  back  and  forth  between  the  icons  and  text.  I  had  to  look  to  the  text  for  the  holding  fix. 
If  you  could  put  that  down  with  the  icons  I  would  never  have  to  go  back  to  the  text.  Up  to 
that  point,  the  prototype  was  great. 

Definitely  the  prototype,  and  I  never  read  the  paragraph,  I  always  looked  at  the  icons.  I  would 
get  the  missed  approach  holding  fix  from  the  plan  view. 


"What  features  of  the  prototype  c/uai  did  you  not  use?" 

Airport  Elevation 

Airport  elevation  -  always  looked  at  airport  sketch. 

Didn't  use  airport  elevation.  Didn't  need  it  in  the  new  location.  Got  it  from  the  airport  sketch. 
Airport  elevation  -  easier  to  go  to  the  airport  sketch. 

Airport  elevation  -  I  looked  at  airport  sketch. 

Airport  elevation  -  I  used  the  airport  sketch. 

Airport  elevation  -  I  went  to  the  airport  sketch. 

Final  Approach  Fix  Information 

[I  would  go  to  the  profile  view  for  the]  final  approach  fix  altitude. 

Got  final  approach  fix  from  profile  view. 

Altitude  at  final  approach  fix  -  when  briefing,  I  looked  at  the  profile  to  find  it.  I  only  looked 
up  top  to  answer  the  first  [quick  reference]  question  to  find  it. 

Easy  to  find  crossing  altitude  [altitude  at  the  FAF]  in  the  profile  view. 

When  doing  approach  after  the  first  [quick  reference]  question  I  would  use  the  profile  view  to 
get  the  first  three  pieces  of  information  [the  frequency  of  the  (VOR  or  NDB)  that  serves  the 
approach,  the  final  approach  course,  and  the  altitude  at  the  final  approach  fix]  on  the  top  line. 
(  Note;  this  comment  addresses  three  information  items.  Therefore,  it  is  also  listed  in  the 
Final  Approach  Course  and  Frequency  of  the  (VOR  or  NDB)  That  Serves  the  Approach 
sections  below). 
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Final  Approach  Course 


For  the  final  approach  course  heading  I  went  to  the  profile  view. 

Got  the  inbound  course  from  the  profile. 

For  ["What  is  the  final  approach  course?"]  I  went  down  [to  the  profile  view]. 

Final  approach  course  [shown  on  the  top  line  in  the  briefing  strip],  I  would  sometimes  go 
down  to  the  profile  view  and  it  is  boxed  down  there  so  I  didn't  need  to  use  the  top  line. 

When  doing  approach  after  the  first  [quick  reference]  question  I  would  use  the  profile  view  to 
get  the  first  three  pieces  of  information  [the  frequency  of  the  (VOR  or  NDB)  that  serves  the 
approach,  the  final  approach  course,  and  the  altitude  at  the  final  approach  fix]  on  the  top  line. 
(  Note;  this  comment  addresses  three  information  items.  Therefore,  it  is  also  listed  in  the 
Final  Approach  Fix  Information  section  above  and  the  Frequency  of  the  (VOR  or  NDB)  That 
Serves  the  Approach  section  below). 

Approach  Lighting 

Also  used  the  airport  sketch  for  the  approach  lighting. 

Freouencv  of  the  rVOR  or  NDB'l  That  Serves  the  Approach 

When  doing  approach  after  the  first  [quick  reference]  question  I  would  use  the  profile  view  to 
get  the  first  three  pieces  of  information  [the  frequency  of  the  (VOR  or  NDB)  that  serves  the 
approach,  the  final  approach  course,  and  the  altitude  at  the  final  approach  fix]  on  the  top  line. 
(  Note:  this  comment  addresses  three  information  items.  Therefore,  it  is  also  listed  in  the 
Final  Approach  Fix  Information  and  Final  Approach  Course  sections  above). 


"Do  you  have  any  other  comments?" 

Briefing  Strip 

Used  entire  top  row  [on  the  prototype].  I  really  like  the  new  top  row  especially.  All  three  top 
rows  are  very  important  for  consistency. 

Really  liked  the  quick  reference  line. 

Used  [approach]  lighting  [sketch]  up  top.  Liked  that. 

Would  prefer  to  see  communication  frequencies  [formatted]  vertical  with  boxes.  Maybe 
because  I'm  used  to  Jeppesen  where  they  are  [formatted]  vertical  in  a  box. 


1-5 


I'm  used  to  Jeppesen  charts  and  they  have  [information  about  the  primary  navaid  that  serves 
the  approach]  at  the  top  of  the  chart.  [On  NOS  charts]  it's  hard  to  find  the  primary  navaid  [in 
the  plan  view]  that  serves  the  approach.  Would  like  to  see  the  improved  [prototype]  version. 
It  stands  out  more.  In  the  new  [prototype]  version  I  like  it  because  it  does  stand  out.  I  used 
the  [approach]  lighting  [sketch]  up  top.  I  liked  that. 

Easy  to  see  left  to  right  [on  the  prototype]  rather  than  vertically. 

I  really  like  the  TDZE  up  top  [on  the  prototype]  -  always  looked  there. 

I  like  having  the  primary  navaid  information  at  top  [of  the  prototype  chart].  I  really  like  the 
way  the  frequencies  are  arranged  [horizontally  on  the  prototype]  much  better. 

I  like  the  lighting  sketch  at  the  top  [of  the  prototype]  too. 

I  continuously  looked  down  on  the  bottom  [on  the  prototype]  for  the  notes.  Probably  because 
I'm  used  to  searching  for  them  on  the  bottom. 

It  is  much  more  useful  seeing  the  touchdown  zone  elevation  next  to  the  airport  elevation  [as 
shown  on  the  prototype].  On  the  NOS  charts,  it  is  a  real  search  to  find  the  touchdown  zone 
elevation. 

I  got  all  information  from  the  new  place  [briefing  strip].  That  was  probably  because  I 
wouldn't  have  to  look  down  as  far  to  the  bottom  of  the  chart,  for  example  on  the  airport 
sketch.  I  could  get  it  right  off  the  top  three  lines. 

Highlighting  Text  with  Bolding  and  Boxing 

Liked  missed  approach  holding  fix.  Like  the  bold  [text].  Would  like  to  see  it  boxed  also. 

Liked  the  bold  print. 

Bolding  didn't  do  anything.  Did  not  really  help.  Approach  course  sticks  out  already  because  it 
was  in  the  middle  of  the  line.  No  need  to  box  it. 

Bolding  [the  MDA  or  missed  approach  holding  point  on  the  prototype]  didn't  make  a  lot  of 
difference  but  [that]  may  be  because  of  the  shutter. 

Frequencies  are  easy  to  read  in  the  bold  [on  the  prototype]. 

Bolded  numbers  are  helpful  for  initial  altitude  for  missed  [approach  procedures]  and  MDAs. 

The  approach  course  I  would  usually  get  from  profile  view.  Really  easy  to  pick  it  up  because 
it  is  boxed  in  profile  view. 

I  would  like  to  see  the  final  approach  course  on  the  plan  view  drawn  out  [highlighted]  more. 
Maybe  more  bolded.  Boxing  the  [final  approach]  course  helped. 
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MSA  Circle 

I  prefer  the  radials  for  the  MSA  [on  the  prototype]. 

For  MSA  circle  preferred  the  text  in  prototype  version,  not  NOS  symbols. 

Prefer  the  prototype  for  the  MSA  circle.  Prefer  the  text  and  sectors.  Helps  you  double  check 
the  primary  navaid. 

I  like  the  radials  in  the  MSA  circle  [on  the  prototype].  I  may  be  used  to  the  old  way,  it  may 
take  awhile  to  recondition  myself  to  do  it  but  I  think  it's  a  better  method.  I  like  the  text  in  the 
center  [of  the  MSA  circle  on  the  prototype]. 

MSA  circle  -  I  prefer  the  bearings.  I  also  prefer  the  text  in  the  center  of  the  circle  [as  shown 
on  the  prototype].  I  like  it  how  it  is  on  the  prototype  chart. 

I  prefer  the  radials  to  the  bearings  [in  the  MSA  circle]  -  it  is  more  intuitive.  [Whether  or  not 
the  symbol]  or  the  text  [was  given]  in  the  MSA  circle  didn't  make  any  difference. 

Iconic  Missed  Approach  Instructions 

Really  liked  the  icons. 
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